Design of the data aggregation platform for a US based real estate service provider while at Planetasia.

The challenge was to build a large scale data aggregation platform that consolidated real estate listings from across the United States. The system had to handle more than 2 million listings on a daily basis.

The system was designed as a distributed workflow with XML processors at each stage. The performance modeling and system sizing done at design time gave the exact characteristics of performance, availability and reliability of the system. The sizing and the scalability model has proved to be valid and exact over the last four years the system has been in operation. By reducing the uncertainty normally associated with performance testing after the system has been built, the project timelines and the effort required were exactly as per plan. Performance testing was automated and required minimal effort.

Next Generation Banking Platform

The platform was designed as a set of small design frameworks - each framework addressing one core design requirement - like configurability, workflow process modeling, communication between components, logging, schedulers, fault-tolerance, revenue management, customer management and so on. All components would integrate with these frameworks, the entire design philosophy was reduced to a set of small computing problems.

The architecture was communicated using a product roadmapping methodology - which uses inputs from various stakeholders, brings all of that together to create long term roadmap. The architecture and Design used the same problem solving techniques and methods. This approach enabled us to evolve a common set of terms of reference for all stake holders.

We created a multi-year product and platform roadmap for both Veloz Global Solutions and for an international bank that serves as the basis for product evolution. The multi-dimensional roadmap addressed the evolution from the perspective of business features, core technology, 3rd party and data source integration, platform applications and platform/product tools.

Each area had further dimensions for instance we had the dimensions of user applications, middleware applications, databases and back-end applications as areas within the applications section of core technology. Each area had a progression path over the series of releases as well as interdependencies with other areas. Related areas of development were picked and formed into a series of platform and product releases. The roadmap was developed with a 3 to 5 year outlook.

The next generation banking platform, the data aggregation solution as well as the Ports and Logistics management applications we developed at Planetasia serve as good examples of architecture driven development integrated into a classical CMM level 4 development methodology.

In both cases, the structuring of the project, the order in which components were designed and built, the test planning and quality processes were driven from an architecture perspective. Both solutions were modeled and built using a distributed workflow system that we designed.

In addition to developing the design, building the data and object models, the interfaces and APIs, we also introduced several quality engineering techniques into the development of these projects. For instance, we had well defined performance and scalability models for the system, we designed APIs to enable automated testing and designed tools to aid in the generation of large quantities of test data. We used iterative development in the creation of certain aspects of the system such as the XML processing engine where we needed to improve the performance of the existing .Net APIs by ten fold. Once the core components were in place, we used a more traditional waterfall model in the configuration of these components for the needs of the customer.