StartingBlocks, our open-source, operational data store product, helps our partners tackle the challenge of consolidating data from multiple sources. By utilizing the implementation of the Ed-Fi data standard and technology suite, EA enables schools and districts to leverage this technology to more easily and cost-effectively integrate data typically siloed within disjointed platforms and disparate software programs into a unified, secure data standard.
To better understand the development and implementation of StartingBlocks along with our decision to make StartingBlocks open source, we talked with Cloud Engineer Manager Eshara Mondal and Vice President of Interoperability Solutions Rosh Dhanawade.
Can you describe how StartingBlocks came to be?
Eshara Mondal: At the end of 2020 and beginning of 2021, we started to have an idea of a hosting product for Ed-Fi. We saw that there was a need in the field for districts to have an easy way to deploy the Ed-Fi standard. Some districts and states didn’t have a great way to deploy Ed-Fi in a cost-effective manner, nor did they have the staff to architect this from scratch.
We had done some work with Ed-Fi in the past, so we had a good idea of what it would take to host and use the Ed-Fi data standard. Due to that previous experience, we found ourselves in a position to advance the option for states and districts. We were equipped with a method to host the Ed-Fi infrastructure in the cloud, so that any education agency that wanted to use Ed-Fi, could use Ed-Fi without the hassle of needing to figure out how to provision their own hardware or resource staff to move their area of expertise into Ed-Fi or cloud hosting.
Principal Cloud Engineer Mark TenHoor found open-source cloud formation templates that had been built by another organization but were not necessarily updated for certain aspects of the infrastructure. For example, we run Ed-Fi on a Postgres backend, which costs less than a Microsoft SQL Server deployment in AWS, so we made some adjustments to the templates so that it would be more cost effective to run in the cloud than the original templates had outlined.
After doing that, we partnered with CORE to identify some smaller districts in California. During this time, we were using Ed-Fi v2.x and currently, we use Ed-Fi v3.x, which is a vast improvement of the application program interface (API) from Version 2. Nevertheless, we deployed Ed-Fi v2.x, which at the time, was just deployable on Windows servers, so we did go the route of having SQL server backends. We set up the infrastructure and started collecting data for districts, but the roadblock we ran into was that we didn’t really have an end product. The Ed-Fi API and operational data store (ODS) lacks visual appeal because it is just an infrastructure offering that lacks a way to utilize that data and visualize that data downstream of the ODS. Its utility is the unification of data and the opportunity to leverage that unification for downstream uses, like visualizations.
Rosh Dhanawade: A good analogy would be that you have the plumbing in your house, but no fixtures. You can effectively move water to all your rooms, but it’s difficult to prove that because there’s nothing to use it.
Eshara: This development was in tandem with our Stadium product. As both were coming online and developing the data warehouse that worked off of the Ed-Fi standard, we were also refining the infrastructure side on the Ed-Fi API and ODS.
Rosh: I’ll also add, a lot of work with Ed-Fi was previously done directly off the database that is the ODS, which then presented issues for scalability and security. If you aim to deploy a solution broadly while relying on API for execution and need to support multiple applications accessing the database directly or providing users with direct data access, you need to navigate key tradeoffs in performance, scalability, and data consistency. The StartingBlocks product is focused on maximizing the throughput of the API to allow vendors to use the API bidirectionally and also use it as a means to get the data to a data store that is more effective for data analysis or advanced data usage. That shift in philosophy was promoted by us and we are still a heavy proponent of that at EA. The API is the main layer of Ed-Fi, not the ODS, and that is one of the core tenets of the StartingBlocks product.
Eshara: It is such a core component to the product that one of the things we really push on is this ability for StartingBlocks to scale. Being that it is a cloud native deployment, we can pull levers that allow the API to scale for state-level or bigger implementations. Often times implementations will have many integrating vendors, and to that end, we needed to make sure that we had many scaling options on the infrastructure side. That’s one of the many reasons we advocate for this approach—we have deep expertise hosting this infrastructure in the cloud. While it may feel like a “black box” to some, our partners trust us to manage this effectively. And so far, we’ve demonstrated that we have the capabilities to deliver. After our work with the smaller California districts, we started a state-wide implementation with South Carolina in 2021. This was a pivotal moment when we began to experience scale, prompting us to refine our infrastructure to support it effectively.
Were there any roadblocks you faced along the way?
Eshara: I think the biggest roadblock was knowledge transfer. On our side, we knew what we were doing, and we knew about the Ed-Fi standard and what it could do. However, to our partners, it was such a novel idea and outside of their current understanding of data systems. We were trying to teach them about using an API properly and the advantages of a data standard that would allow for interoperability. It was less about the infrastructure and the technical side (though we still had to address technical problems as they came up) and more about the theory behind it. South Carolina is a great partner because they took a step into the unknown, not knowing what the outcome was going to be, and they allowed us to come in and share our expertise so that we could grow something together. Now in the third, going on the fourth year of integration and we’re getting to a point where data is moving away from their old pipelines towards the new standards and it’s starting to prove out really well.
Rosh: I think another significant shift during the past year and a half has been rethinking how to evolve the product—not as a complete transformation, but as a strategic shift. The focus has been on balancing its role as infrastructure while ensuring it provides enough utility for partners to manage it effectively—not entirely on their own, but at least 60% independently. This utility helps us, internally, because we can use those same resources that are doing those support activities to drive even more impact and do even more product development. One of those shifts was thinking through what are the commonly asked tasks that we have a cloud engineer or interoperability engineer do to assist someone and how does that impact their workflow? We wanted to get over the challenges, so with StartingBlocks, we aimed to provide users with greater ownership. Initially, there was no utility or administrative application that allowed them to manage key functions independently, and we recognized the need to bridge that gap. For someone to take more ownership of the infrastructure and to be able to administer it themselves, we had to take a step to then provide them access to that. About a year and half ago, the Ed-Fi Alliance released the Ed-Fi Admin API. We also received funding from Michael & Susan Dell Foundation (MSDF) to create what we now call the StartingBlocks Admin App, or SBAA.
That enabled partners to operate independently and feel like they have more ownership of the infrastructure. Before that, it was, “Okay, we know that EA is managing these things for us, and we go to EA for answers.” Now, it has become, “EA makes sure that the infrastructure is running, and they’ve given us the ability to do the work that we need to do and to turn around the requests that we may get from districts or partners that we serve and turn that around much more quickly.” It’s not necessarily any difference in how the Ed-Fi application is hosted, but it’s more that we must look internally and ask, “What could make this better?” and also, “What can enhance the lives of the partners we work with and also lead to more entrenchment of the StartingBlocks product?”
Previously, it functioned as a hidden framework, and now, with an application layer, partners can use detailed insights into the data being ingested. It also enables self-service capabilities, such as managing credentials and generating access keys that can be shared with vendors to leverage the same infrastructure. The application has reduced the amount of EA support hours, which has allowed us to make StartingBlocks open source. All these little additive effects of the decisions we’ve made with StartingBlocks have been driven by our desire to ensure that people feel like this is their infrastructure. EA serves as a managed service provider, but this is our partners’ data infrastructure, and we want our partners to have ownership of all dimensions of their data. This enables us to focus on the things that we do best, which is continuing to make that infrastructure more feature-rich while also adopting and making sure that we’re deploying future versions of Ed-Fi and then delivering that to our partners with as little friction as possible.
Eshara: Previously, without the admin application, it was truly a black box. With the development and implementation of the admin application, it provided our partners with insight into the infrastructure. It gave them new knowledge points to digest and understand Ed-Fi. We also wanted to open-source the codebase to demystify the system—it’s not magic; it’s just code that anyone with relatively minimal understanding of AWS could deploy themselves. Since we continuously update the application and keep it aligned with the latest Ed-Fi standards and API changes, we believe we’re in the best position to serve as a trusted source for maintaining and evolving this technology. We’ll always be there to support and publish that code base so that they know they’re getting it from a trusted source.
Can you expand a bit about that and why making StartingBlocks open source is also part of EA’s values and how we’re contributing to the larger ecosystem? I’d like to hear both of your words about the importance of that.
Rosh: One goal at EA is to provide education entities with as much freedom of choice as possible. Part of that freedom of choice means providing a head start for those without the technical expertise or staffing to build from scratch. By offering a ready-to-use foundation, we help organizations source talent, upskill their teams, or reallocate existing technical staff to leverage what’s already been built—saving time and resources while allowing for customization and growth. When looking at open-source technologies, there is deep thinking and lots of architectural work involved that would take another agency months, but more realistically, years, to complete. Developing and implementing something like StartingBlocks involves 8- or 9-years’ worth of aggregate knowledge and expertise. Most public agencies don’t have the ability to spin that up or dedicate that much time to creating infrastructure from scratch. One of our overarching goals is to ensure that every student and educator in the country has access to high-availability, high-fidelity analytics. To achieve this, we must also empower them to make informed financial decisions—enabling cost control over infrastructure so that funding can be strategically directed towards areas that require unique investment.
Right now, many schools and districts face uncertainty, unsure of what they can manage in-house versus what they need external support for. By making our software widely available, we aim to equip educators with the knowledge they need to allocate funding effectively, avoiding unnecessary roadblocks and maintain momentum in delivering high-quality analytics to their classrooms. I think that’s the big picture. We do this because we see this gap between a desire to have rich, analytical environments in PK-12 and the concerns regarding, “How do we do it?” and “Are we paying the right amount of money to do this?” We hope to shorten the gaps in as many of those spots as possible by making as much of the technology as transparent as possible, but also as performant as possible. One thing that undergirds both StartingBlocks open source and the Enable Data Union (EDU) is that both of these tools are not just proof of concepts.
An education agency can take these tools, deploy them, and immediately access a fully-fledged, enterprise-grade data infrastructure. This approach bridges the gap between understanding where to invest resources and implementing and managing a working solution. This is the core philosophy behind EA’s commitment to open-source technology for K-12: empowering education agencies to move from “This is a good idea” to “This is an idea in action.” We enable data unification, giving agencies the freedom to scale solutions that work best for their unique needs.
Eshara: Open source is a way to establish a community and establish a conversation with other folks that want to utilize our code base. It opens the discussion of, “Hey, did you think about changing this in your code base?” or “Hey, did you know that doing X would make it more performant?” We want to have that conversation with folks who are using the code base, so that we can establish a standard to grow the code base into something that is better for everybody involved.
At EA, we’re very lucky that we have a wide view of the field. If a smaller district wants to use our code base, but there’s three different things we would need to change to meet their needs, we are happy to have that discussion and we’re happy to change the code base in such a way that it would be applicable to particular use cases as well. That is the idea around open source—we’re sharing this with the world, and we want to hear what the community has for feedback to solidify our codebase as an acceptable standard.
When you look at our industry, are we the vanguards? Are we leading the charge? Where is EA in this space?
Eshara: In some ways, yes, because we’re a non-profit. We have the mission to do good and not solely focus on profit. I think that is a motivating factor and also a big reason why we can open source our code base. We want to demystify some of the technical elements that are behind the curtain so the schools, districts, and partner organizations that we work with know we protect their data and show them how.
Rosh: Within the realm of education technology, we are leading the way as it relates to making accessible, high-quality tools. Similar efforts exist in other domains that embrace this same approach. While we may not yet operate at the same scale or impact, we need something akin to the Linux Foundation. The Linux Foundation ensures that the Linux and Unix code bases remain open, transparent, and subject to rigorous scrutiny and collaboration.
We aim to foster an ecosystem where openness and shared innovation benefit the entire education sector. Even within the Ed-Fi space, EA is unique in our commitment to making both the tooling and expertise widely accessible, ensuring that more education agencies can leverage and contribute to stronger, more connected data infrastructure.
There are countless times the StartingBlocks team has shared their insights, knowledge, and findings back to the community, either through tickets to the Ed-Fi Alliance or presentations. The goal is not just the creation of artifacts, but also the sharing of knowledge. Our approach to knowledge sharing is proactive—we surface insights and others may not know to ask for, and we introduce ideas that spark new conversations, setting a precedent in K-12 EdTech and paving the way for others to follow.
Eshara: Accessibility is key here. While the Ed-Fi API and ODS are technically open-source and available for use, their complexity can be a barrier. StartingBlocks simplifies this by adding an abstraction layer, so users don’t need deep expertise in Ed-Fi’s inner workings. Instead, they can deploy our tooling with confidence, knowing it’s built on our experience and continuous engagement with the community.
What are you most proud of about this process and what you’ve accomplished?
Eshara: I’ve learned a lot about the inner workings of different education agencies, and I’ve learned more about the technical details of Amazon Web Services (AWS) and various coding idiosyncrasies. I’m also proud of our team and how we’ve worked together to build something like this.
As Rosh mentioned, I don’t believe any other organizations have achieved what we have. I’m proud of our team for working together, finding those pain points, and building solutions.
Rosh: For me, it’s one thing to claim that you are trailblazers, but it’s a whole other thing to actually prove as such. If we had to pinpoint one driving force behind our decision to make an open-source solution, it would be the culmination of years of work. The final push to release it was remarkably fast—compressed into just a few weeks.
We had an interesting conversation at the Ed-Fi Summit with one group that was saying, “Hey, it’s really hard for us to evaluate how to work with Ed-Fi because we have to spend time to spin it up and get it running and even if we get it running, we don’t know if it’s actually going be similar to what we encounter in the field.” That was our first opportunity to after we released the open-source repository to say, “Hey, you don’t even need to have any kind of relationship with us, you can just go to this repository, pull it down and have something that you can deploy that gives you as close to a production-ready environment as possible.” Even if folks are not going to be a provider of Ed-Fi, if they’re going to be a consumer or they’re going to send data to Ed-Fi, having a test bed to push that data to and to be able to have realistic feedback from the environment and know points at which it scales and points at which it doesn’t is super valuable.
We want more vendors to consider Ed-Fi and consider native integrations. It benefits districts and education agencies because they now have more choices.
Our original vision was that we wanted people to have more choices, while at the same time, be able to make informed decisions, so I’m incredibly proud of being able to say, even to a competitor, “Take our stuff out for a spin, maybe you can contribute and you can realize where our gaps are and you can help inform that, or you can realize where you can improve your own product.” Both of those outcomes are totally fine with us.
Eshara: I will add, I think there’s a question about why didn’t we do this sooner? I think if we had done it sooner, there would still a question in my mind about:
- Is what we’re offering feature complete so that an agency could just take it off the shelf, deploy it, and have something working?
- Is it proven? Does it work in the field? Have we used it in the field in product instances and is it working?
We got to a point where Mark TenHoor and I felt very comfortable—we’re feature complete, we have all this tooling around our stack, where if we put it out there and you’re an infrastructure engineer and you deploy this, or if you’re an IT administrator with AWS access and you deploy this, you have all the tools you need to make sure that your infrastructure is working properly. You have all the tools that you need to make sure that your API is working properly, your database is working properly, and all the monitoring tools are there too, which is the icing on the cake for some of this stuff.
Dare I ask, what’s next?
Eshara: We want to just keep moving forward. The Ed-Fi Alliance is always enhancing the API and ODS code base and I think one of their newer developments is completely changing how it works, so it’s going to be an effort, a research and development effort, on our end to realize that vision from the perspective of StartingBlocks. That’s probably the biggest thing on the horizon. I’m also excited to see the community uptake of this and see whatever comes out of that and what the engagement looks like there.
Is there anything we didn’t ask?
Rosh: Our call to action for this blog is to do what we said: investigate the repository, plug in, and become part of that community that we’re trying to build. It’s a daunting task to figure out how you get community off the ground, so we want to say that we’re here. If you look at the code and if you’ve deployed it, we want to hear from you and we want to know what additional questions you have. Are there any issues that we haven’t addressed with our documentation? All of that feedback is welcome, and we would love to engage with folks.