Buildmasters

The search for a C# DAL code generator that actually works. Part 3: the solution.

tubby taylor — Thu, 04 Sep 2008 08:28:44 +0000

In part 2 of this series, I outlined my requirements for an ideal DAL generator.

Well, not having found any tool out there that meets my criteria, I ended up writing the tool myself, meeting all but one of my objectives [I dont handle stored procedures yet].

Sql Server 2005 is the target DBMS for now, using VS2005 as the compiler.

Lets walk through how to use the tool.

From the command line, the tool is invoked like this.

mycodegentool.exe config.xml

The simplest config.xml file is



 D:\development\codegenExample\myDatabase

 Data Source=.;Initial Catalog=myDatabase;

    Integrated Security=SSPI;

 update_date

This will give you a DAL dll with every table and view of the database in it, and handle concurrency for any table that that has an update_date field.

To get a unit test dll as well, add this xml fragment to the config file.



 D:\development\codegenExample\myDatabase\CodeGenUnitTest

This takes care of the plumbing. Now reference the DAL dll from your business layer dll, and happy porcelain coding.

Here are a few use case scenarios in actually using the DAL from your business layer.

Load up a table.

ProductsFactory factory = new ProductsFactory();

List rows = factory.LoadAll( connection ) ;

Console.WriteLine( rows.Count ) ;

Quicker way to see have many rows in a table.

ProductsFactory factory = new ProductsFactory();

int rowcount = factory.GetRowCount( connection ) ;

Load up a specific row from a table using the primary key.

int id = 17 ;

ProductsFactory factory = new ProductsFactory();

Products product = factory.LoadByPrimaryKey( connection, id ) ;

Load up a set of rows from a table based on custom search criteria.

bool discontinued = true ;

ProductsFactory factory = new ProductsFactory();

List rows = factory.LoadByDiscontinued( connection, discontinued ) ;

To add custom search criteria to a table or view, add this xml fragment to the config file.

where the contents of the sql file are

where discontinued = @discontinued order by productname

Get a row’s parent row.

Here we are walking the employee hierarchary using ootb generated methods - cool.

EmployeesFactory factory = new EmployeesFactory();
// robert - the pleb

int? employeeId = 7;

Employees robert = factory.LoadByPrimaryKey( connection, employeeId);

Assert.AreEqual( robert.FirstName, "Robert" ) ;
// robert reports to steven

Employees steven = robert.parentEmployeesByReportsTo( connection ) ;

Assert.AreEqual(steven.FirstName, "Steven");
// steven reports to andrew

Employees andy = steven.parentEmployeesByReportsTo( connection);

Assert.AreEqual(andy.FirstName, "Andrew");

// andrew is the big cheese Employees nevilleNobody = andy.parentEmployeesByReportsTo(connection); Assert.IsNull(nevilleNobody);

Get a row’s children rows.

Here we start with an order, then go up to get its customer, then go down to get all the orders of this customer - powerful pattern in a few lines of code.

OrdersFactory factory = new OrdersFactory();

int orderId = 10289 ;
// get an order

Orders order = factory.LoadByPrimaryKey( connection, orderId); 
// the customer for this order

Customers customer = order.parentCustomersByCustomerID( connection) ;

// all the orders of this customer List orderList = customer.childOrdersByCustomerID( connection ) ;

Load up from a view.

Same as loading from a table.

Alphabetical_list_of_productsFactory factory = new Alphabetical_list_of_productsFactory();

List rows = factory.LoadAll( connection ) ;

Load up from a custom query.

Same as loading from a table, with some extra config.

CustomerOrderFactory factory = new CustomerOrderFactory();

List rows = factory.LoadByRegion( connection, "Quebec" );

List rows2 = factory.LoadByCustomer( connection, "quick" ) ;

The config in this case would be

Here we have a customer order view, for display on a web page data grid for example.

The CustomerOrder.sql file contains

select * from customers c, orders o

whilst the LoadByCustomer method has the following criteria in the config sql file reference

where c.customerid = o.customerid and c.customerid = @customerid order by c.companyname, o.orderdate

and the LoadByRegion method’s where clause is

where c.customerid = o.customerid and isnull(c.region, '') = isnull(@region, '') order by c.companyname, o.orderdate

Insert a new row

OrdersFactory factory = new OrdersFactory();

Orders order = new Orders() ; // create a new row in memory

order.CustomerID = 123;

order.EmployeeID = 7 ;

order = factory.Save( order, connection ) ; // does the insert

Update an existing row

OrdersFactory factory = new OrdersFactory();

int orderId = 10289 ;

Orders order = factory.LoadByPrimaryKey( connection, orderId); // get an order

order.ShipPostalCode = "2565"

order = factory.Save( order, connection ) ; // does the update

Insert then update a row

OrdersFactory factory = new OrdersFactory();
Orders order = new Orders() ; // create a new row in memory

order.CustomerID = 123;

order.EmployeeID = 7 ;

order = factory.Save( order, connection ) ; // does the insert

order.ShipPostalCode = "2565" order = factory.Save( order, connection ) ; // does the update

Delete an existing row

OrdersFactory factory = new OrdersFactory();
int orderId = 10289 ;

Orders order = factory.LoadByPrimaryKey( connection, orderId); // get an order

order.MarkForDelete = true ; factory.Save( order, connection ) ; // does the delete

Negative use case - get a concurrency error.

TestTableFactory factory = new TestTableFactory();
TestTable alice = factory.LoadByPrimaryKey( connection, 12345 ); // get a row

TestTable bob   = factory.LoadByPrimaryKey( connection, 12345 ); // get it again
alice.update_date = DateTime.Now ;

bob.update_date   = DateTime.Now ;

alice = factory.Save( alice, connection ) ; // does the save ok bob = factory.Save( bob, connection ) ; // will throw a concurrency exception

Negative use case - get a save error when saving a stale object

OrdersFactory factory = new OrdersFactory();
int orderId = 10289 ;

Orders order = factory.LoadByPrimaryKey( connection, orderId); // get an order
order.ShipPostalCode = "2565"

factory.Save( order, connection ) ; // does the update - ok

order.ShipPostalCode = "2000" factory.Save( order, connection ) ; // will throw a stale row exception

Many users simultaneously inserting rows into a table.

In this example, a total of 5000 rows will be inserted into a table by 50 threads running in parallel. The table in question has a monotonically increasing non-identity integer ID primary key field, and therefore it is a contention hotspot.

In the first version of this use case, the inserts fail, as the code and sql that determine the next primary key value are not protected against concurrent threads. The test fails with the error “Violation of PRIMARY KEY constraint ‘PK_TestTable3′. Cannot insert duplicate key”.



protected void InsertLotsOfRowsInParallel()

{

 using ( MyThreadPoolManager tpm = new MyThreadPoolManager( 50, 5000 ))

 {

  for (int i = 1; i <= rowsToInsert; i++)

   tpm.Queue( InsertOneRow() ) ;

  tpm.WaitUntilAllFinished() ;

 }

}

// every invocation of this method runs in its own thread

protected void InsertOneRow()

{

 using (TransactionScope transaction =

         new TransactionScope(TransactionScopeOption.RequiresNew))

 {

  using (SqlConnection connection = new SqlConnection( _connectionString ) )

  {

   connection.Open();

   TestTable3Factory factory = new TestTable3Factory();

   TestTable3 obj = new TestTable3();

   // BAD: get the next available ID = max(ID) + 1

   obj.ID = select max(ID) + 1 from the table

   obj.update_date = DateTime.Now;

   // insert the row - this will fail with PRIMARY KEY constraint violation

   obj = factory.Save(obj, connection);

   transaction.Complete(); // commit - wont get here

  }

 }

}

In the correct version of this use case, the codegen tool has ootb capability to handle this common scenario. The tool will escalate the lock around the next primary key value and serialise access to the table, giving everyone a warm fuzzy feeling.



// every invocation of this method runs in its own thread

protected void InsertOneRow()

{

 using (TransactionScope transaction =

         new TransactionScope(TransactionScopeOption.RequiresNew))

 {

  using (SqlConnection connection = new SqlConnection( _connectionString ) )

  {

   connection.Open();

   TestTable3Factory factory = new TestTable3Factory();

   TestTable3 obj = new TestTable3();

   // GOOD: the tool will get the next available ID, the safe way

   obj.ID = factory.GetNextId( connection, TestTable3.Column_ID, true );

   obj.update_date = DateTime.Now;

   // insert the row - now it works

   obj = factory.Save(obj, connection);

   transaction.Complete(); // commit - aha

  }

 }

}

Here are a few .NET Reflector screen dumps of the DAL layer that the tool generates for Northwind.

This is what the Orders row object looks like.

And the corresponding factory object for the Orders table.

Here we have a custom query object that is a join between Customers and Orders tables.

And the corresponding factory object.

This screen shot shows some of the generated code involved in unit testing the DAL. Here, in the insert phase, each table is being populated and saved in referential integrity order, with pre and post row counts, object state, and field values being asserted.

One of the databases that I have recently codegened is the popular DotNetNuke database.

You can download my DAL for DNN here and also download the corresponding unit test dll here.
I recommend .NET Reflector to inspect the generated code.

Contact Buildmasters if you would like a DAL generated for your database.

This is the to do list for the tool

Make sure it compiles and works under Mono and Debian Linux.
Make sure it compiles and runs under VS2008.
Extend to target SqlServer 2008, MySql, Oracle databases.
Extend to handle stored procedures.

So far I have used the tool against 13 corporate, vendor and public databases, and it has my made my programming experience easier, faster, and more reliable.

I do not think about the DAL layer at all now - it just works.

ShareThis

The search for a C# DAL code generator that actually works. Part 2: the requirements.

tubby taylor — Thu, 04 Sep 2008 07:57:02 +0000

Now that I have outlined the reasons why current DAL tools do not satisfy me, I will list my requirements for the ideal DAL code gen tool.

The tool should:

Generate the DAL for the whole database in one hit, not table by table, generating all non-system tables and views out of the box.

Have a simple pleasant configuration file experience.
Ideally, the minimum configuration required would be a connection string to the database, and a directory path to which files are generated.

Be able to support the addition of custom query objects and custom criteria clauses, and to do so in an easy manner.

Create the DAL in a separate dll, so that the source code of my business rules layer contains only business rules code.

Be a command line tool, so I can run it as part of my build process, and via cruise control, regenerating the DAL every night and on-demand during the day, keeping it in synchronised with the underlying database schema.

Create objects that do not hold any database locks.
Therefore all objects returned from the DAL are to be disconnected.

Be designed with concurrency in mind from the ground up.
No joke, the vast majority of systems that I have encountered over the last two decades have not taken concurrency into account.
Consider this scenario: Alice and Bob both read row A from a table in the database.
Alice modifies her copy of row A and saves it back to the database.
Bob then changes his copy and successfully saves it back.
Congratulations, your database is now corrupt.
This is incorrect behaviour, as Bob should have received a concurrency exception when he attempted to save his stale row.
The DAL tool should support this out of the box.
This is plumbing - the programmer should not have to worry about it.
Concurrency handling is especially important now that we live a disconnected data set world.

Uses nullable types. I do not want to explicitly check for nulls before I get or set a field.
The tool should do it.

Not use events.

Be designed with database transactions in mind from the ground up, and should work with implicit transactions via TransactionScope, or explicit BYO transactions.

Generate clean, understandable code that is easy to follow and debug.

Generate deterministic predictable code, so that small schema changes do not cause large diffs.

Use the referential integrity of the database itself to automatically generate accessor methods on objects that allow you to walk the database tree.
These accessors to parent and child objects are to be lazy populated, ie when the accessor is invoked..

Have a simple object model using the factory pattern.
TRow : RowBase , List, TFactory: FactoryBase.

At buildmasters, we have a very strong test ethic.
Even generated code needs to be tested.
The tool should auto generate unit tests to code cover the generated DAL code.
The unit tests should perform the following tasks in one database transaction:
- Insert a row into every table, populating each field, read the rows back, and assert the row counts and field values.
- Update the inserted rows, changing the values of each field, read the rows back, and assert the row counts and field values.
- Delete the inserted rows, and assert the row counts.
- Rollback the transaction [unit tests should leave no footprint in the database].

Have the following persistance patterns:
1. Successfully persisting a row object invalidates the object, so it cannot be saved again, and returns the latest version of the row in a new object.
2. Persisting an unmodified row object does nothing.
3. If a table does not have a primary key, its row objects cannot to be persisted.
  This is a classic case of bad database design. Every table needs a primary key.
4. A standard persistance mode that follows the rules of database integrity:
  A new memory row object can only be inserted, not updated, based on primary key.
  A row object that is read from the database can only be updated or deleted, based on primary key and concurrency cookie, and only if the concurrency cookie is still valid.
5. An ETL persistence mode:
  Persisting a row object involves trying an update, and failing that, an insert is attempted, all based on primary key.
  The concurrency flag is ignored.
  This mode of operation is used in batch ETL type tasks where you just want to transfer or inject a whole bunch of rows in to a table, whether that table is in the same database or not, and maybe on a different connection.

Provide a DAL wrapper around legacy stored procedures.
Im not a big fan of TSQL [see my post here ].
I have seen many instances of whole systems that have been written in TSQL, where the business rules have been buried away in unmaintainable several-thousand-line TSQL stored procs, as opposed to having the business rules in a proper SOA/business tier written in a high level language like C#.
It is a pain to debug TSQL and it does not scale.
But sometimes you will inherit such a mess, so at least the DAL tool can hide it behind a nice C# interface.

In part 3 of this article, I will outline a tool that meets these requirements.

ShareThis

The search for a C# DAL code generator that actually works. Part 1: the problem.

tubby taylor — Thu, 04 Sep 2008 07:55:34 +0000

Besides developers, the database is your company’s greatest asset.

The data in your database, your ability to get at it, shape it, analyse it, report on it, turn it in to understandable meaningful information, is what gives your company competitive advantage.

I have been designing, using, optimising and programming against relational databases for over 20 years. DB2, Oracle, SqlServer, Sybase, Access, and MySql are some of the major relational databases engines that I have worked with, and DAO, ADO, ADO.NET, visual studio dataset designer, nHibernate, CSLA, codesmith, and LINQ are some of the tools and DAL technologies that I have used to get at the those databases.

In all that time, I have been frustrated with all of the above DAL methods of accessing data, for various reasons that I will discuss below.

I found that I was spending too much time stuffing around trying to placate or work around the DAL technology and not enough time just getting the job done.

Developers should spend their time writing the porcelain (the business rules that are specific to solving a business problem that cannot be generated by a tool) and very little time writing and working around with the plumbing (the repeatable monkey code infrastructure that can be generated by a tool).

Let us take the visual studio dataset designer for example.
The visual studio DAL tool is fairly good, but I have several issues with it which have caused me a great considerable frustration .

For one, it does things one table at a time. This is very cumbersome when one has 100+ tables in a database that needs to be DALd.

It is also very slow. When I right click on the designer, my pc freezes for several seconds until the context menu pops up. FWIW I have a high spec dev box.

The generated code does not compile in SQLCLR safe mode.
Why can’t something I generate in visual studio work in SQLCLR safe mode?
Hello Microsoft. Visual studio and Sql Server are both your products.
The problem is that the designer inserts a bunch of delegates and events into the generated code, and events use synchronisation objects, which are forbidden in SQLCLR safe mode, and there is no way to prevent generation of these events every time you touch the designer.
Who actually uses those events anyway?
Am I the only person that thinks that events are overrated and over used.
For app support teams, events are a nightmare to debug.
One needs innate familiarity with the code to visualise the execution flow, even when using a debugger.
In my experience, events have been more trouble than their worth over the years.

Null checking. The code generated does not use nullable types for some reason, so every time I get or set a property, I have to write code to first test for null, otherwise the dataset code will kindly throw an exception.
I just want to set a property – 1 line of code, not 5, please.

Also, after your program is complied, you don’t know if or when the underlying database schema has changed. Guess when you find out – runtime!
I want the DAL layer to be in sync with the actual underlying database schema at all times.
If I had a command line interface to the visual studio dataset designer, I could regenerate the DAL layer through my continuous build process and have confidence in the accuracy of my DAL.

Another annoyance is that sometimes the designer changes column orders, seemingly at random, and a small change in the designer causes illogical reordering in the designer.cs file, making diffs hard to read.

The typed data adapter classes do not expose their contained SqlCommand objects publicly, so I cannot associate an explicit transaction to them.

The other data access methods don’t do the job for me either:

CSLA: the version I used was based on XSLT templates – OMG -–enough said.

nHibernate: getting closer, doesn’t handle my requirements see below.

LINQ: ditto.

In part 2 of this article, I will outline my requirements for an ideal DAL code generator.

ShareThis

Legacy MetaData Analysis

adam — Fri, 22 Aug 2008 06:09:00 +0000

Do you have complex legacy systems with extensive database components? Do you need to map out what you have in order to move forward efficiently?

This kind of analysis is time consuming and error prone if attempted manually. We have a fast and efficient automated solution that will save you time and money during this type of project.

Some of the issues your Business Analysis Team will face are:

Legacy System with 000’s of Tables, Views, Stored Procedures and Functions
Complex interdependencies between them
No documentation and no original developers
No-one knows what is being used and what isn’t
Fragile environment as small changes in one place can have large impacts elsewhere

The Buildmasters MetaData Analysis process uses a combination of industry standard tools and our expertice to generate a comprehansive and easy to navigate resource that will enable your own team perform their analysis.

SEE A SAMPLE REPORT HERE

An engagement with us involves the following non-intrusive steps:

1. Proof Of Concept

This is a low cost exercise that allows us to sample the source database systems and produce a report covering a fragment of the databases for appraisal by the client. The process will typicall involve:

Extraction of database structure from source database(s)
Parsing and Cleansing of Metadata
Load into the Buildmasters Metabase for integrity checking and further cleansing
Processing via the Buildmasters Database Documentation Tool
Publishing of Documentation either online or in a portable help file

____________________________________

2. Comprehensive Data Analysis Report

Having refined the output during the Proof-Of-Concept the full report is generated and cleansed by the Buildmasters Team before being delivered to the client for final acceptance. An example of one of our meta-data analysis reports is available here, generated using the sample Northwind database.

The Proof-Of-Concept (POC) step guarantees our clients expectations are met before having incur significant analysis costs. If you could like to find out more about this service please contact us using the link below.

ShareThis

Scrum Secret #2 - low tech rocks!

adam — Sat, 02 Aug 2008 03:12:34 +0000

There are a number of elements to SCRUM that really work well for us. One of the most obvious features is our ‘SCRUM Wall’, where we ‘publish’ all the work items that form part of the current sprint to ‘The World’.

The Wall is primarily for the team, but it’s surprising how many ‘outsiders’ take a real interest. Initially there’s the comments about ‘nice wall - but a bit low-tech isn’t it’, to which we respond - ‘that’s the point’. The great thing about the Wall is that each card is a tangible thing, representing a tangible deliverable. Rather than being locked away in a spreadsheet or web app (though we do use TACKLE to track our tasks electronically to - more on that later) the cards can be accessed easily by everyone.

The cards typically begin life with a one-liner describing a deliverable, and estimate of how long it will take and the initials of one of the developers. However it doesn’t take long before the card has evolved with the scribblings of several members of the team as more information is discovered.

Quite often one card will become two or three as the iterative process of understanding more about the task unfolds. The pure simplicity of the wall means that things don’t get forgotten or fall off the radar - every thought and new discovery is captured quickly and easily.

The wall only represents 2 weeks of work at a time - a Sprint. This means that every 2 weeks the wall gets cleared. The completed tasks are archived. The items that are not complete or blocked are taken down and form the basis of the ‘Backlog’ for the next sprint, in addition to new scope items that we take from our ‘product Backlog’.

We are currently running a one-hour planning session with the whole team every 2 weeks, which gives us ample time to review where we are with the existing work items and allows us to estimate how much new scope we can bit off to keep us occupied for the next 2 weeks.

This process is incredibly tranparent and efficient. The constant replanning and course correction means that: we rarely build anything that is not part of the final solution; everyone on the team is cross skilled and across the big picture; we can easily slot in refactoring tasks to increase the robustness of components we have already built but need some ‘love’; reporting upwards to management on progress is easy due to the high visibility of the methodology (lets face it - who could miss our scrum wall!).

The best thing about this methodology is that the team is completely behind it. The core team members don’t want to work without it and new recruits accept and join in with the processes eagerly.

LONG LIVE SCRUM!

ShareThis

Application virtualization with xenocode

grendo — Mon, 14 Jul 2008 13:25:48 +0000

Application is the future of virtualization. Xenocode allows you to package up an application and deploy it in a virtaul exe to run anywhere, (anywhere on windows). I came across it when I wanted to create a portable application using the ,net framework. Xenocode is amazing. It allowed me to package up a winform application written in C# to produce a stand alone exe that could run on any windows box even without the .net framework installed. I have not tried it under wine/linux but it is on my todo list.

While we have yet to use it in a project think how easy a deployment process will be when have xcopy deployment without any need for a framework to be installed on the destination machine.

So for all you .net coders out there start pumping out the usb portable apps with visual studio.

http://www.xenocode.com/

turn your dotnet exe into a portable usb app

ShareThis

Subversion

grendo — Mon, 14 Jul 2008 11:53:52 +0000

Subversion is our prefered tool for version control. With the latest version you have merge tracking and can compare history of word and excel documents, (when using the tortisesvn client). With this new functionality I can see it being our document management system as well. It is fast, easy to set up and transactional, (commits all your changes or none at all unlike pvcs). As a added bonus it is opensource and free to use.

http://subversion.tigris.org/

There are many clients for subversion the two we prefer are

http://tortoisesvn.tigris.org/

http://www.visualsvn.com/

This is one of the core bits of software you want running on day one of a project. We had a architecture team try to force us to use pvcs which is non transactional and pay for the privilege. What a step backwards that would have been. It was only that we had it running in our build process from day one that stopped this happening. It was a bit of a lesson for me, if you want your key bits of technology supporting the development process have them running on day one. I wish we did the same with our bug tracking tool. Unfortunately we were forced to use trackrecord which I would not recommend to anyone, (especially when there are some many excellent open source bug tracking tools).

ShareThis

The New Developer

grendo — Mon, 14 Jul 2008 11:31:53 +0000

We believe that when a new developer starts work in our team they should be able to start work on real project tasks by lunch. To achieve this you need a number of things to be in place namely

A Buildprocess
Automated unit testing
Developers virtual image
Project tasks that can be allocated to a new developer
A developer allocated to pair with the new developer in the initial stages.

Added to this we have a number concepts to keep in mind

Developers are drip fed into the team. New developers should be assimilated into the team one or two at a time. This is to avoid old school or negative views around the benefit of automated builds and unit testing. By having one or two developers come in at a time it is easy to get them coding and testing the way we want.

Egoless programming. By this we mean no developer owns a specific piece of code and you should always be able to listen to a different view. Don’t be defensive when someone when someone offers a different approach to something. I have met very few developers I could not learn something from, (they do exist but usually will not fit into ego less programming because they have big ego’s).

Everyone is Expendable. If you find someone in your team that can not be replaced easily start to arrange so their knowledge gets shared around. The reality of the IT industry is people change jobs all the time. The culture we have built up in our team has kept this to a minimum as everyone enjoys the success and development processes we follow. From a management point of view we have done a good job if on any given day we can loose an experienced resource and the team keeps working as usual. Having one person own a price of code can have disastrous results. If one person is left to be the only developer on a piece of the application they generally fall back to their core skills to solve a problem. A recent example we encountered was when we inherited an etl process. We had no influence over the development and were lead to believe that ssis was used to implement it as prescribed by the project architecture. After we looked at the code we soon realized that just about all of the etl was implemented in tsql which was the developers primary skill. All the out of the box functionality which makes ssis a useful etl tool was ignored as it was not in the skill set of the developer. This left us with a very complex solution to something that should have been a lot easier if it was implemented by someone with ssis experience. In the end we had to make the call to rewrite the etl layer using ssis functionality and keeping the design simple.

Be wary of people who tell you they are an architect every chance they get. While some architects are worth their weight in gold they are few and far between. Make sure you check out their cv references through people you know instead of who is on their cv. I onced asked a architect who was running a data warehouse project if their database was a typical data warehouse schema expecting something like start or snowflake schema but got “Yes third normal form”.

Application support should be involved in the development process. Instead of getting new resourses it is better to get application support staff involved in the development process. How better way to ensure your legacy of automated builds and deploys will be continued when the project moves into the application support phase.

Keep It Simple. Every project is a battle to keep technology decisions simple. I have worked on very few large projects that have not been over engineered to some degree. You are always battling architecture groups, enterprise teams, empire builders and people who can talk well in meetings but are light on technical ability to implement a simple solution. You loose your fair share of these battles due to political influences. The only thing you can do in this case is to make sure you have a review at the end of the development cycle where you point out to upper management how a simple job was made complex to the detriment of the project. If you can hire someone off the street with 2+ years experience, they can sit down and understand your code, maintain it and deploy changes to production you have done your job.

So when a new developer arrives on site what happens first. During their first hour

They meet the team during our morning 10 minute scrum. During the meeting they are assign a task.
We pair a developer with them
They copy our developers vmware image to their computer
Start the developers image
Run a script to personalise the image
Start our build process and run the local clean step. This will download the latest databases, code and binaries that successfuly built on the build machine, (compiled and pasted unit tests).

The developer now has everything they need to complete their task and become a valued contributor to the team in no time at all.

ShareThis

64 bit Windows: WTF: 64 bit dlls live in System32 dir, 32 bit dlls’s are in SysWow64

tubby taylor — Mon, 30 Jun 2008 11:39:35 +0000

Here at Buildmasters, we have now moved all our development vm’s and other servers to 64 bit editions of Windows.

The 4GB memory limit is gone, and no more recycling IIS app pools and running out of virtual memory.

One thing to note however: The \System32 directory contains 64 bit dlls, and the legacy 32 bit emulation dll’s live in the \SysWow64 directory.

The is all done for backwards compatibilty, but it is very confusing.

Have a look at http://en.wikipedia.org/wiki/WOW64
for more info.

ShareThis

SqlServer2005: Move From TSQL To SQLCLR And Save Your Project

tubby taylor — Tue, 24 Jun 2008 14:12:53 +0000

Many organisations have lots of business rules in TSQL stored procedures.
You’re right, that’s crazy but that’s what we see all the time, and our current project in Sydney, Australia using the Eagle PACE product is no different. Eagle PACE is an Opertaional Data Store for the financial services industry, in our case the Funds Management sector. At it’s heart is an enterprise database with a relational data model appropriate to the industry, and is surrounded by data feed mechanisms which in basic terms are a series of Stored Procedures.

Whilst a TSQL stored procedure is good for simple logic, eg stringing together a couple of selects and inserts, it is simply not up to the task of handling complex business rules and workflow.

I would say that a TSQL stored proc of more than 200 lines is just to complex to debug, enhance and test.

TSQL falls short in the following areas:

1) Code reuse.
There isn’t any.
No classes, no rowtypes, no inheritance of stored procs or functions.
No way to encapsulate functionality - the only way is, that’s right, you have to write another stored proc and have nested calls - yuk!
Absolutely awesome when you have 133 parameters to pass around.
[Dont laugh - we have this many parameters and more in the stored proc interfaces we have to maintain]

2) Error handling.
SqlServer2005 has try/catch, which is good, but not yet widely used.
Most TSQL code we see has lots of if/goto statements after each real code statment.
This blows out the size of the proc and turns in into spaghetti.
This kind of error handling is quite intrusive, so one loses track of what the storeed proc is actually doing due to all the eror handling code.

3) Testability and code coverage.
It is hard to unit test and do code coverage on a large stored proc.
You cannot break it down and test chunks of it in unit tests.

Here’s a scenario we see all the time.
You’ve been handed a 2000 line stored proc and it has a bug or it needs an enhancement.
There is no documentation of course, so to fix the problem you have to figure out what the hell it does, starting from the top and following the code paths until you find the problem.
Oh dont forget all those goto error statements.
Oops, I’ve just been distracted because the PA just walked past my desk, and I’ve lost my place somewhere in the bowels of this multi-page proc, so I have to start again at the top to figure out where I have to make the code fix.

At buildmasters, we were presented with several dozen very large TSQL stored procedures that were written in other countries [some of the worst TSQL code I have ever seen I must add] that we had to maintain and enhance [no documentaion provided of course]. All part of an Eagle PACE consulting enegagement in Sydney, Australia.
These stored procs were unmaintainable.
It was clear that we would not meet our development deadlines if we did not innovate at the TSQL level.
Not only this, but the vendor’s product (Eagle PACE) could only talk to external components via ODBC, so refactoring into a C# or Java business layer was out of the question.

The innovation we came up with was to write/refactor all TSQL stored procedures and functions into C# SQLCLR.

Not only did we get to write business logic in C#, we were able to abstract much of the code into base classes and helper methods.
This, together with the use of C# generics, nullable types and predictable try/catch error handling, meant that we wrote a lot less code.
We literally reduced the code base by 1000’s of lines by moving from TSQL to SQLCLR.
Less lines of code means less bugs, because, as we all know, coding is the art of inserting bugs into software.

Now, a new developer can approach an unfamiliar SQLCLR stored proc with confidence that he will be able to understand its purpose within a short time.
Because common functionality is encapsulated in base classes and helper methods, a stored procedure’s code consists of only the specific business rules required for its purpose.
It is much easier to find a bug or the place where an enhancement is to be performed.
Error handling is consistemt and non-intrusive - just throw an exception and let the base class do the rest.
Also, developers can unit test SQLCLR C# methods and base classes in isolation.

We also wrote a deployment tool that takes a C# SQLCLR assembly and registers the appropriate stored procedures and functions into the SqlServer database, so that as far as Eagle PACE is concerned, our SQLCLR stored procs appear as normal TSQL stored procs, happily callable from the vendor’s ODBC interface.
We have innovated the Eagle in a way that has never been done before.

This is done automatically as part of our continuous build and deploy process, and all these C# SQLCLR stored procedures are subject to the same unit test code coverage as the rest of our code.

SQLCLR - really cool technology.

ShareThis