Market Data Monopoly? - Part One
May 7, 2009
The provision of accurate and timely market data is truly the lifeblood of almost every financial services organisation worldwide. Every day these orgaisation rely on the latest stock prices, financial instrument attributes, corporate action events, rates and analytics in order to make trading decisions, reconcile their accounts and track their performance.
These requirements for a steady stream of vast quantities of data have give rise to a number of extremely successful market data organisations who collate their data from the exchanges and brokers and package it up so it can easily be consumed by banks, investment managers and anyone else who can afford to sign up for an account.
Two of the major players in the market data business are Bloomberg and Reuters. Their market data products are very similar and allow their clients to subscribe to market data feeds via several mechanisms, including ‘Pay-As-You-Go’ whereby you can make requests for information (eg prices and attributes) about a particular financial instrument (eg BHP Ordinary Shares) and they send back the details, for which you pay ‘Per Security’ - currently around $2(USD), depending on security tyep (Bond, Equity, Derivative, etc). Alternatively you can subscribe to entire segments of security data in a ‘batch’, for instance ‘Asian Equities’, and receive the entire universe of those securities daily. The choice of mechanism really depends on how many securities of a particular type the client needs data for and what the most cost effective mechanism is at the time. Read more
Tales from the War Room
April 29, 2009
With out 3rd major software release due in 40 days success is really reliant on some pretty effective collaboration between the multiple moving parts that need to come together on the day like one big well-oiled machine. That’s why I’m so glad that we setup our ‘War Room’ meeting. Each morning at 9am 20 indivduals descend on Meeting Room One from their various teams and take stock of progress over the previous 24hrs.
From the top we have : the Delivery Manager, ultimately responsible for the successful implementation of the release into Production; the Project Manager, reponsible for coordinating the campaign and keeping us all revved up; the Test Director and test team leads, responsible for executing 500 test cases with 100% success rates through System Test, Joint System Test and Joint Acceptance Testing; the Trading System BA, responsible for design and development of the BlackRock Trading Platform component; the Data Management System team lead, responsible for the 40+ integration components they feed data throughout the solution; the Enterprise Services team lead, responsible for scheduling of data feeds; the Environments team lead, reponsible for servers, hardware and dealing with the spate of City-wide power-cuts we’ve had recently!!! Finally there’s the ‘Stats Guy’, responsible for compiling an impressive array of graphs and statistics that boil down the infite number of variables that all the other members generate during any given day into something that can meaningfully be discussed in under 45 minutes. Read more
Code Branching gone mad!
March 23, 2009
Question : What happens in the last year of a multi-year, multi-million dollar, multi-release program of development and business change???
Answer : All the hard stuff that has been left to the end has to be delivered in parallel, in overlapping releases and in double quick time before the money runs out…oh, and if you’re really unluck you’ll have to upgrade some of the applications from the earlier releases too.
This is a reality for us at the moment on our project at one of Australia’s largest fund management companies. Despite shifting requirements and architecture we have managed to deliver 2 successful major releases over the last 2 years and yet our biggest challenge lays ahead of us.
So far we have built a powerful Investment Data Management Platfor using the Eagle PACE product as a base, into which we have integrated accounting feeds from Hiport, market data feeds from Bloomberg, Factset and S&P. On the other end we have integrated 3 trading platforms - Charles River (for Equities and Enterprise Compliance), Imagine (for Derivatives) and BlackRock Aladdin (for Fixed Income). On top of all that we have integrated a Performance and Anlaytics system (StatPro), Recs System (Smartstream TLM) and Corporate Actions system (Smartstream Coacs). To really finish things off we have added a Data Mart layer to support Internal reporting via Business Objects and External Reporting via SS&C Pages - PHEW!!
How can there be more you’re asking yourselves? Well now we have to feed Registry Data into the mix too from multiple registries and switch off several legacy systems ( not forgetting remediation of a reasonably long list of small access and excel based apps).
Finally we are upgrading Eagle, Charles River, StatPro and SS&C Pages….all before December 31st 2009!
As you can imagine we have alot of issues to overcome, and there’s not enough room on this page to discuss them all, but I thought I’d share with you one of the many issues that just get overlooked when these gran plans are hatched - code branching!!
For the uninitiated code branching (and merging) is the situation you find your self in when you have multiple release dates of the same code base running concurrently. In a perfect world you would only have 2 lines of code - Development (known also as ‘THE TRUNK’) and Production Support (known also as ‘THE PROD BRANCH’). In this perfect world when ‘hot fixes’ are applied to the ‘PROD BRANCH’ they are then merged into ‘THE TRUNK’ to keep it in line with with what is in Production, that way when a major release is deployed to Production from THE TRUNK it includes all the hotfixes that have been accumulated since the last major release.
Our problem is we have mul;tiple teams working on multiple ‘major’ and ‘minor’ releases of the same application (The Data Management Platform). As a result we have THE TRUNK, THE PROD BRANCH and 2 additional branches - one for an upgrade release and another for a parallal major release. Confused? I’m not surprised! Here’s a photo of the whiteboard after our team meeting this morning..
The release order runs left to right. The PROD BRANCH is always on the left as a deplpoyment to Production by the support team can happen anytime, next comes ‘91′ which represents an upgrade of our data management platform toolset to version 9.1, then there’s 3A - the current major release we are working on - due to go to Prod in June. Finally there ‘4′ that is due to go to Prod in Septemberbut development has to start now - hence another branch!!!
The arrows at the top indicate where all the overhead exists with this situation. Changes in earlier branches must be merged into later branches to keep them up-to-date. The can be quite time consuming - particularly as some of our components (in the ETL space) are binary and can’t be auto-magically merged by our source code repository tools (Subversion).
We aren’t exactly sure what the impact will be on our project plan - we intend to perform the merge tasks as we go to avoid breaking huge numbers of our automated unit tests. A cursory estimate is indicating about an additional 20% to each development task - a total of 1 man month in total. I’ll be talking to our PM today - hope he’s in a good mood!!
Scrum Master
A Lesson From Kalashnikov
November 26, 2008
Ok, don’t panic, we haven’t gone gun crazy, but we picked up an interesting analogy today from one of our business sponsors. It went something like this : “The Kalashnikov AK47 only has 13 moving parts and is the most successful weapon ever invented, remember this when you start designing your next software delivery. Keeping things simple is the key to building something that is reliable, will accept the abuses imposed by our operating environment and will stand the test of time”.
This is a more colourful way of repeating our mantra “Keep IT Simple Stupid”. Applying this ‘KISS’ principle has enabled us to deliver robust and reliable software into a production environment over and over again. More importantly that software is Maintainable and Extensible - qualities that are often forgotten by the project team, most of whom have no intention of hanging around after ‘Go-Live’.
If I were to pick the biggest challenge we have faced on our current project it would be the transition of project to ‘Business As Usual’. Despite a large comms budget and endless workshop sessions in an attempt to engage the future custodians of our systems it has been an uphill struggle. Our team has suffered less than others, despite being one of the larger elements of the overall program, and the one thing that has made the process more palatable is the fact that we have tried to reduce complexity at every opportunity. If I were to list the top 5 tips for ‘Keeping It Simple’ they would be :
- Don’t accept the early version s of the solution architecture without thorough review. It has probably been conceived from a 60,000ft view and is likely to be over engineered. Review it, Challenge it, Simplify It
- Any steps that result in writing less code is good, as long as it is not at the expense of true requirements. Less code means less testing and fewer points of failure.
- Keep the number of ’systems’ involved in an end-to-end process as few as possible. Each step in the chain is a potential point of failure, and with different systems using different logging and exception mechanisms means the process of diagnosing issues becomes a phorensic excercise of huge proportions.
- Standardize your development process and avoid ‘Specialists’. Each member of your design, development and testing team should be as interchangeable as possible. They should subscribe to a common set of goals and processes, and should collaborate regularly. If a team member is working in isolation for long periods of time you can bet he/she is building a problem.
- Integrate small chunks of functionality at a time. It’s much easiler to resolve issues and refactor components if you and introducing small changes to the code base regularly rather than attempting large integrations. Your options are severley limited if you find problems after a large integration (in terms of time and resources), especially if a signisifcant redesign is needed.
Scrum Secret #2 - low tech rocks!
August 1, 2008
There are a number of elements to SCRUM that really work well for us. One of the most obvious features is our ‘SCRUM Wall’, where we ‘publish’ all the work items that form part of the current sprint to ‘The World’.
The Wall is primarily for the team, but it’s surprising how many ‘outsiders’ take a real interest. Initially there’s the comments about ‘nice wall - but a bit low-tech isn’t it’, to which we respond - ‘that’s the point’. The great thing about the Wall is that each card is a tangible thing, representing a tangible deliverable. Rather than being locked away in a spreadsheet or web app (though we do use TACKLE to track our tasks electronically to - more on that later) the cards can be accessed easily by everyone.
The cards typically begin life with a one-liner describing a deliverable, and estimate of how long it will take and the initials of one of the developers. However it doesn’t take long before the card has evolved with the scribblings of several members of the team as more information is discovered.
Quite often one card will become two or three as the iterative process of understanding more about the task unfolds. The pure simplicity of the wall means that things don’t get forgotten or fall off the radar - every thought and new discovery is captured quickly and easily.
The wall only represents 2 weeks of work at a time - a Sprint. This means that every 2 weeks the wall gets cleared. The completed tasks are archived. The items that are not complete or blocked are taken down and form the basis of the ‘Backlog’ for the next sprint, in addition to new scope items that we take from our ‘product Backlog’.
We are currently running a one-hour planning session with the whole team every 2 weeks, which gives us ample time to review where we are with the existing work items and allows us to estimate how much new scope we can bit off to keep us occupied for the next 2 weeks.
This process is incredibly tranparent and efficient. The constant replanning and course correction means that: we rarely build anything that is not part of the final solution; everyone on the team is cross skilled and across the big picture; we can easily slot in refactoring tasks to increase the robustness of components we have already built but need some ‘love’; reporting upwards to management on progress is easy due to the high visibility of the methodology (lets face it - who could miss our scrum wall!).
The best thing about this methodology is that the team is completely behind it. The core team members don’t want to work without it and new recruits accept and join in with the processes eagerly.
LONG LIVE SCRUM!
64 bit Windows: WTF: 64 bit dlls live in System32 dir, 32 bit dlls’s are in SysWow64
June 30, 2008
Here at Buildmasters, we have now moved all our development vm’s and other servers to 64 bit editions of Windows.
The 4GB memory limit is gone, and no more recycling IIS app pools and running out of virtual memory.
One thing to note however: The \System32 directory contains 64 bit dlls, and the legacy 32 bit emulation dll’s live in the \SysWow64 directory.
The is all done for backwards compatibilty, but it is very confusing.
Have a look at http://en.wikipedia.org/wiki/WOW64
for more info.
Eating the Elephant
June 12, 2008
Our current engagement with a funds management company in Sydney, Australia is a large and complex beast, involving more moving parts than a swiss chronometer, an incredibly demanding business and the Eagle PACE Operational Data Store. The entire program will run for approximately 4 years, involve over 100 consultants, vendors and staff, and burn over $100 Million. We are interfacing with 3 front office trading systems (Charles River, Imagine and Blackrock Alladin), several downstream systems (Smartstream TLM, CoAcs and StatPro) and exchanging market data with Bloomberg, Reuters, Factset and S&P - Phew!
It’s taken nearly a year to truly appreciate the size of our ‘Elephant’ and learn that there’s only one way to eat it…one small piece at a time. Short term, realistic goals are the key to success, which fits beautifully into our development methodology - SCRUM. Read more
“The dates can’t change but the scope can and will”
June 5, 2008
It’s the age old problem of managing change effectively on an IT project. The business have expectations of what functionality will be delivered, the joint testing with external parties has been locked in and the go-live weekend has been booked BUT the detailed requirements are incomplete and the functional designs are only half finished (at best).
Of course some contingency was built into the original schedule, but that was 3 months ago and the world has changed alot since then.
The same thing happens every time, just to different degrees, it’s just the nature of our business. The key to success in my view is not to get too bogged down and stay ‘Agile’. The fact is that no one is really to blame, the project plan is just a guide. If the dates can’t change and the scope creeps then the inevitable result is that what gets delivered on the ‘immovable’ date is not quite ready, a little under-cooked. Is this a problem? in some cases yes, but how big a problem can be managed.
At the end of the day the business is the one that gets affected - but remember, the immovable dates and incomplete requirements are ultimately down to the nature of THE BUSINESS and the ability of THE BUSINESS to be able to state their requirements accurately in the first place.
We’re all in it together, business and IT. When things don’t go to plan we make a new plan. It’s as simple as that!
Release 2 Mayhem
May 29, 2008
Another crazy week is coming to an end with one of Australia’s biggest funds management companies. Our project plan is finally level and we are about to go full steam into developing a series of new interfaces between our Eagle PACE database and the new derivatives trading system (Imagine).
Our plan to refactor our ETL layer from R1 is also well under way with only a few components still using the old design - putting us in an excellent position to decommision ‘ETL1′ by the end of June.
Our SCRUM development methodology is keeping us focussed and well organised. Sprint 5 started on Monday, running for two weeks until 13th June (when we make another delivery of R1 warranty fixes to Production). The juggling of R1 waranty work and new R2 development is continuing to keep us on our toes and further reinforcing our decision to go with a very agile approach to running the team.
