Web Development Bonanza
May 29, 2008
It seems that the demand for new website development has not been subdued by the downturn in the Australian economy. Our phones and mailboxes are stuffed full of requests for new site development, existing site enhancement and search engine optimisation. Long may it continue!!
Release 2 Mayhem
May 29, 2008
Another crazy week is coming to an end with one of Australia’s biggest funds management companies. Our project plan is finally level and we are about to go full steam into developing a series of new interfaces between our Eagle PACE database and the new derivatives trading system (Imagine).
Our plan to refactor our ETL layer from R1 is also well under way with only a few components still using the old design - putting us in an excellent position to decommision ‘ETL1′ by the end of June.
Our SCRUM development methodology is keeping us focussed and well organised. Sprint 5 started on Monday, running for two weeks until 13th June (when we make another delivery of R1 warranty fixes to Production). The juggling of R1 waranty work and new R2 development is continuing to keep us on our toes and further reinforcing our decision to go with a very agile approach to running the team.
Enterprise Reporting
May 28, 2008
After conquering the challenges of the Operational Data Store (ODS) our next logical hurdle was always going to be dealing with the inevitable Reporting requirements that would follow. To make things more complicated there always seems to be more than one ODS involved, as well as numerous legacy data sources that need to be brought into the picture.
We have been fortunate enough to be able to work with the some excellent Data Architects and have refined our approach over the last few years, helping us pick the RIGHT SOLUTION from the numerous options that alway seem to present themselves.
ETL Components
May 28, 2008
Buildmasters have been involved with moving data from A to B for a number of years and have seen all manner of ETL architectures and designs. One thing we have learnt is that no implementation is ever the same, however there are a series of common principles that should be applied in any ETL project in order to guarantee success and avoid delivering something into production that is a monster to support. Read more
Build Process
May 28, 2008
The heartbeat of any Buildmasters project is our continuous build process. This combination of technologies allows a team of developers to develop and unit test the code confidently and in isolation whilst the entire process of integrating the completed components is automated on our ‘Build Machine’.
Each time a developer checks in a piece of code the Build Machine detects the change in our Subversion Source Control System and kicks off an automated build of the entire project using Visual Build Pro. We follow a Test Driven Development (TDD) approach, which means that each code component that is checked into source control is accompanied by a number of unit tests also. The automated build compiles the code for the new component and the unit tests and then runs the tests. Read more
Enterprise ETL - Keep It Simple Stupid
May 28, 2008
I wish I had a dollar for every time I see a consultancy roll in with a development framework of one sort or another that is the answer to all our problems. More often than not these cobbled-together code-bundles are merely a foot in the door for bringing in legions of more consultants to utilise the framework to build the solution. It’s all a big con. The value added by these frameworks is often less than the value of building the solution from scratch.
Personally I always apply the KISS principle. Keep It Simple (Stupid!). If the solution isn’t lean and straight-forward then it is likely to be more trouble than it’s worth.
Generally a simple ETL design can be applied for the majority of development. Ensuring that adequate coding standards are developed in conjunction with templates, can help with code consistency, simplicity and maintainability.
