The good news: The online education company was revolutionizing the industry and growing like gangbusters, providing a virtual education experience to thousands of students across the country.
The bad news: The company’s 10 year-old systems were sputtering and stalling, unable to keep pace with the company’s explosive growth, and the online school found itself offline one too many times.
The problem: Everyone and everything was calling on a single, enormous database. It was used by every user for every action, hosting millions of live testing sessions, calculating zillions of test scores, performing institutional reporting, and everything else.
Imagine all of Target’s check out registers in all its stores attached to a single database that also served the corporation’s inventory management, time card, security, and corporate reporting functions. Scanning one too many items would overload the system.
The same was true for this online school. One test taker could crash the system trying to register a True or False answer because she was sharing the same resource at the same time as a teacher adjusting a grading scale against hundreds of thousands of test scores.
After a decade of rapid growth, the company had reached a tipping point where the push of a button really could break everything.
The Data Dilemma
The original system was designed much like an online version of the old, one-room schoolhouse: students of all ages attended school in one classroom, at the same time, with one teacher to juggle all the learning activities.
But now this online school had too many students for its one room and one teacher to handle. Management sought the quickest solution: bigger hardware, with more memory and more CPUs. But hardware merely enlarged the metaphorical classroom. It didn’t create separate classrooms for different functions. All the work continued in one, albeit larger, database. Still a single schoolroom with a single teacher, stretching thinner and thinner with each new student.
Spotty systems performance was causing bad user experiences and threatened to damage the company’s reputation and impede its growth. A technology overhaul was overdue.
This one room schoolhouse would need to be transformed into a modern campus to serve the volume and complexity of its rapidly expanding student body.
The original system had been designed appropriately for its time, when success was uncertain and resources scarce. But the one-database-fits-all scenario had gone as far as it could.
The company knew it had been borrowing against the future, choosing fastest implementation over best system architecture. And it had worked! The strategy allowed the company to grab winning market share and perfect its business model. But now the time had come to pay down its technology debt.
The transformation involves teasing apart a decade of fastest-not-best, carefully replacing it with best-not-fastest, and adding a healthy dose of future, all while this merry schoolhouse continues operating and growing at full tilt.
It requires a major commitment, is expensive to execute, and will not happen overnight. And that’s okay! Because this little red schoolhouse won. It attracted a large student body and now has the resources to make things right.
No longer will overburdened systems hobble performance or threaten this company’s future. Instead, a scalable infrastructure, built for the long-term, will power the next generation of growth.
1. Growth has consequences
Balance resources and demands today against what they may be tomorrow. You can borrow against the future, and often are wise to do so. Just know that if your firm survives and thrives, eventually you will have to pay that debt.
2. Timing is everything
Your success could be your downfall if you miss your timing. Massive growth brought system failure during peak season for this education company. You can push your tipping point only so far without risking catastrophic failure or a miserable slide backward while the market overtakes you.
3. Recognize limitations
Hardware doesn’t fix software problems, it just moves the tipping point further along, obscuring the underlying issue. You wouldn’t add an engine to a tricycle when what you really need is a car.
4. Later costs more
It costs 10 to 100 times more to change software after it’s operational than when it’s being built, so construct it as forward-thinking as you can. The more you do, the closer your cost multiplier will be to 10 than 100 when it’s time to pay down your own technology debt.
Originally published in SmartCEO January 2015