I recently spoke at the Neolane Evolution conference in Paris about how my company evolved our Marketing Dialogue Manager™ distributed marketing platform from an underperforming, unmaintainable monolithic beast into a modern modular system that integrated with Neolane’s marketing automation capabilities. Interestingly, many months before I had started writing a blog post about the very same topic. Here it is, all freshened up.
When we started, our platform consisted of a single massive codebase that tried to do everything. By “everything”, I mean the user interface was talking directly to the database, handling all of the business logic itself. The system also included marketing automation, content management, digital asset management, workflow management and PDF generating functionality. I think it might even have made lattes.
It didn’t do any one of these things terribly well.
At the time v1 was conceived, the build vs. buy analysis done by my predecessors pointed to “build”. This seemed rational at the time because no single product out there did everything in the way that our clients needed. Regrettably, the ambition of the endeavor outweighed the time and expertise we had to execute a system that was competitive and met our clients’ growing needs. Every time we touched one part of it, another part was affected. Regression defects were a constant issue and any fix or improvement took forever.
With any big problem, the best approach is to break it into smaller more manageable parts. So the first thing we needed to do was split the application into smaller components.
First, we acquired a 3rd party Variable Data Printing (VDP) platform for handling printed marketing materials.
This gave us a great deal more capability as well as more control over the quality of our final outputs. Then, we offloaded marketing automation capabilities to Neolane. These were major projects, but relatively seamless from an end user perspective. Most of the work was happening in the background. With each step, we offloaded limited functionality from our own homegrown system to systems that are consider best in class at what they do.
Once we had all of the major foundational components reliably humming away in the background, we turned our attention to the core distributed marketing activity management functionality of our platform. We began building an all-new Marketing Dialogue Manager™, using new technologies and an all-new user experience designed to help non-marketers engage with their customers in a fun and easy way, supported by agency-grade strategy and analytics.
When we did build the new MDM, we also implemented an enterprise service bus (ESB) to act as an abstraction layer between each major component of the system. This meant that no system was directly integrated with another; all the integration happened in the ESB. Those familiar with enterprise software architecture will know that ESBs exist to protect systems from each other when one system needs to be upgraded or replaced. Instead of having to update every single system that talks to the module that’s getting changed, only the ESB has to be updated.
This approach also helps with migration from one module to another; once integration testing has been completed, the ESB lets you essentially throw a switch that will activate the new module and deactivate the old one with minimal downtime.
From beginning to end, this transformation has taken about three years, so it’s not quick. But for us, taking a big, unwieldy, monolithic system and selectively upgrading its parts has proven to be an effective way of getting ourselves out of an expensive and untenable situation.