Astoria and Jasper: Old Ideas New Technology

SubSonic is an excellent open source DAL generation tool developed by Rob Conery that has been around for almost a year. In most cases SubSonic can just be added to your project, pointed to a databse, and you magically and immediately gain access to a rich and strongly typed object model that can be used to query and persist data to and from your relational data source. SubSonic also has a cool feature called the REST handler that essentially allows one to use HTTP protocol as an interface to data stored in a back-end relational database. It is URI based and returns data back in XML format. This data can then be used by decoupled client applications as they see fit. If you have never looked at SubSonic, you owe it to yourself to checkout this very cool tool. Recently Microsoft unveiled their plans to release similar features codenamed Astoria and Jasper with .NET framework 3.5 and Visual Studio 2008. The CTP of Astoria and Jasper are available for download from Microsoft. Jasper is described by Microsoft as:

Project Jasper is geared towards iterative and agile development. You can start interacting with the data in your database without having to create mapping files or define classes. You can build user interfaces by naming controls according to your model without worrying about binding code. Project Jasper is also extensible, allowing you to provide your own business logic and class model. Since Project Jasper is built on top of the ADO.NET Entity Framework, it supports rich queries and complex mapping.

Pablo Castro, the mastermind behind Astoria describes it as:

The goal of Microsoft Codename Astoria is to enable applications to expose data as a data service that can be consumed by web clients within a corporate network and across the internet. The data service is reachable over HTTP, and URIs are used to identify the various pieces of information available through the service. Interactions with the data service happens in terms of HTTP verbs such as GET, POST, PUT and DELETE, and the data exchanged in those interactions is represented in simple formats such as XML and JSON.

The Astoria web site also includes sample online services that showcases how this new technology can be used. It also allows anyone with a Passport account to design and host their own experimental data services .

Microsoft Build Engine (MSBuild)

Up until Visual Studio 2005 and .NET Framework 2.0, the actual build process of solution or project files was pretty much a black-box phenomenon for developers. With .NET Framework 2.0 and Visual Studio 2005, Microsoft unveiled its new build platform called MSBuild. MSBuild essentially provides a transparent build process through Visual Studio IDE, as well as allows developers to build projects and solutions from the command line. This allows us to fully customize our builds and create builds on machines where Visual Studio is not even installed. On a recent team project I was using continuous integration or CI (an Agile practice) via CruiseControl.NET. It was a breeze to automatically build projects and solutions using MSBuild on our build server (a run-of-the-mill Dell workstation running Windows XP without Visual Studio 2005 installed) whenever someone checked-in any files to source control. This allowed us to be confident at all times that various pieces our distributed project integrate well. Moreover, since MSBuild can build projects and solutions from command line, I had some batch files setup that would build release versions of our projects, create setup files (using NSIS), archive older versions of setup files, and move new setup files to a network share where MIS / Operations folks could install the new version on the production server from. In Visual Studio 2005, project and solution files are nothing more than MSBuild XML build scripts. This allows us full control over the build process. For example, in a web project, Visual Studio 2005 does not provide any user interface to modify pre and post build actions. However, since solution file is simply an MSBuild script, we can modify such actions manually by opening up the solution file in any text/XML editor. MSBuild can perform several key tasks out of the box. Each task is essentially a unit of work (UOW) that contributes to the entire build process, e.g. copy files and folders, compile files, etc. If we ever require tasks that do not ship with MSBuild, we can always create custom tasks in any .NET language by simply implementing the ITask interface or deriving from the Task class. In conclusion, MSBuild allows developers full control of the build process so that builds can be fully customized. This may not be so crucial in smaller projects, but for enterprise solutions that typically comprise of several projects and many dependencies, MSBuild is definitely a God-send!

ISP: Interface Segregation Principle

The Interface Segregation Principle (ISP) deals with cohesion and is closely related to the Single Responsibility Principle (SRP). In Uncle Bob's terms, the ISP states that:

Clients should not be forced to depend on methods that they do not use.

In other words, this principle states that if a class performs multiple functions that may be used by different client classes, then those functions should be seperated out into different interfaces according to their usage. The idea behind this principle is to confine the aount of rework to only clients that use a certain interface, should the interface of a depended-upon class changes.

MSF Agile v4.0 Team Model

Out of all the Agile methodologies I have studied so far (XP, Scrum, AUP, and MSF), I really like the fact that MSF provides a Team Model that goes into details about how project teams should be structured. While it may be obvious to some, I know from experience that properly structuring teams is not something that that comes naturally to many organizations. In fact some organizations have less success with agile only because of their ineffective team structures. MSF Team Model is based on the following basic principles:

• A team of peers with clear accountability, shared responsibility and open communications. Each role is accountable for a specific share of the quality of the overall solution.
• Advocacy for all key constituencies that must be represented on a successful software project. Every perspective is represented to provide the checks and balances that prevent errors of omission and lopsided decisions.
• Stretch to fit to the scale necessary for the specific project. Constituencies may be combined in small teams or further refined as teams scale for larger projects.

There are 7 advocacies that are represented in the MSF Agile Team Model. In my opinion it's imperitive that these advocacies have proper representation in order to ensure success of a given project. Since MSF offers a stretch-to-fit approach, it's possible to map these 7 advocacies into fewer role clusters as determined by the phyical size of the team. Following are the advocacies and their respective focus as presented by the MSF Team Model:

Product Management

Goals

• Understand, communicate, and ensure success from the standpoint of the economic customer requesting the solution.

Functional Areas

• Marketing
• Business Value
• Customer Advocacy
• Product Planning

Responsibilities

• Acts as customer advocate
• Drives shared project vision/scope
• Manages customer requirements definition
• Develops and maintains business case
• Manages customer expectations
• Drives features vs. schedule vs. resources tradeoff decisions
• Manages marketing, evangelizing and public relations
• Develops, maintains, and executes the communications plan

Program Management

Goals

• Right solution is delivered at the right time and all expectations are understood, managed and met.
• Deployed solution will meet qualities of service & business objectives, and be viable in the long term.

Functional Areas

• Project Management
• Solution Architecture
• Process Assurance
• Administrative Services

Responsibilities

• Drives development process to ship product on time
• Manages product design and specifications
• Facilitates communication and negotiation within the team
• Implements and ensures standards
• Maintains the project schedule and reports project status
• Drives implementation of critical trade-off decisions
• Develops, maintains, and executes the project master plan and schedule
• Drives and manages risk assessment and risk management

Development

Goals

• Implementation, estimates, high quality maintainable code and unit tests.

Functional Areas

• Technology Consulting
• Implementation of Architecture and Design
• Application Development

Responsibilities

• Specifies the features of physical design
• Estimates time and effort to complete each feature
• Builds and/or supervises building of features
• Prepares product for deployment
• Provides technology subject matter expertise to the team

Quality Assurance

Goals

• Ensure solution quality.

Functional Areas

• Test Planning
• Test Engineering
• Test Reporting

Responsibilities

• Ensures all issues are known
• Develops testing strategy and plans
• Conducts testing
• Reports test results

Release/Operations

Goals

• Timely readiness and compatibility of infrastructure.

Functional Areas

• Infrastructure
• Support
• Operations
• Commercial Release Management

Responsibilities

• Act as advocate for operations, support and delivery channels
• Manage procurement
• Manage product deployment
• Drive manageability and supportability trade-off decisions
• Manage operations, support, and delivery channel relationships
• Provide logistical support to the project team

User Experience

Goals

• Provide user documentation and training
• Understand and communicate users’ context, and ensure usability from user perspective

Functional Areas

• Technical Communications
• Training
• Usability
• Graphic Design
• Internationalization
• Accessibility

Responsibilities

• Acts as user advocate on team
• Manages user requirements definition
• Designs and develops performance support systems
• Drives usability and user performance enhancement trade-off decisions
• Provides specifications for help features and files
• Develops and provides user training

DIP: Dependency Inversion Principle

The Dependency Inversion Principle deals with how to correctly design classes such that their dependency on one another causes the least amount of work in case of a change. Uncle Bob's definition of DIP states:

• High-level modules should not depend on low-level modules. Both should depend on abstractions.
• Abstractions should not depend on details. Details should depend on abstractions.

Essentially what this principle means is that in a tiered design, higher level modules and lower level modules should not directly depend on each other; instead they should only depend on abstractions. Moreover, abstractions should not have any knowledge of details (other classes in the system). This principle closely relates to some of the other design principles I discussed previously in that it yields a design that is highly decoupled ensuring that minor changes in one part of the application do not cause a domino effect in other parts of the application. Moreover such a design is extensible -- as new business entities are added, usually such as design scales well by requiring only new code to be added rather than existing code to be modified. ASP.NET 2.0's Provider Model is a great example of such a design that provides a default implementation for various sub-systems such as Membership, Personalization, Navigation, etc. but also allows developers to modify the default behavior by extending certain classes.

LSP: Liskov Substitution Principle

The Liskov Substitution Principle dictates when and where it is correct to derive a subtype from an existing supertype. As defined by Barbara Liskov and Jeannette Wing, it essentially states that:

Let q(x) be a property provable about objects x of type T. Then q(y) should be true for objects y of type S where S is a subtype of T.

What it means is that if S is a subtype of T, then a function q(x) must behave in the same manner irrespective of the type of x whether it be S or T. In other words, if a piece of code behaves differently for a subtype than a supertype, then that code violates the LSP (and consequently the OCP). Let's say that we work for a bank and that we have a simple teller application. Currently the teller application works with checking and savings account types. public abstract class Account { public abstract double CurrentBalance { get; } public abstract void Deposit(double amount); } public class CheckingAccount : Account { private double _currentBalance; public override double CurrentBalance { get { return _currentBalance; } } public override void Deposit(double amount) { _currentBalance += amount; } } public class SavingsAccount : Account { /* Savings account implementation */ } Our bank just entered the mortgage business so we need to modify our teller application to support this new account type. During the short analysis phase we decide that since mortgage account is type of an account (IS-A relationship), we can simply create a new mortgage class deriving from the Account abstract class, override a method and a property, and we should be in business. We add a new class as follows: public class MortgageAccount : Account { private double _currentBalance; public override double CurrentBalance { get { return _currentBalance; } } public override void Deposit(double amount) { _currentBalance -= amount; } } Everything sounds logical, that is until we come across this code in the teller application: public class Bank { public void ReceiveMoney(Account account, double amount) { double oldBalance = account.CurrentBalance; account.Deposit(amount); Debug.Assert(account.CurrentBalance = oldBalance + amount); } } Based on our pre-existing code, the ReceiveMoney() method is making a reasonable assumption that the new balance should be equal to the old balance plus the newly deposited amount. This assumption is, however, violated if we pass an instance of the Mortgage class to this method. This is a clear violation of the LSP.

Agile Resource Allocation

For Agile methodologies to really work with the promised increase in efficiency, it's important that all participating team members be generalists rather than specialists. For some companies migrating to Agile from other traditional methodologies, this may not be the case. I have worked with companies that have well-defined and assigned roles for Business Analysts, Database Administrators, Front-end Developers, Back-end Developers, etc. When such companies move towards Agile, they face a problem of optimal resource allocation and utilization. As an example, a developer may not be productive while a DBA is creating new tables in the database and writing stored procedures. Once the DBA is done with his tasks, he may not have enough to do until the start of the next iteration. Since Agile is iterative, and iterations can't overlap, this may result in poor resource utilization and may adversely effect the overall efficiency. Before starting to implement an Agile methodology within your organization, it is important to make sure that your IT human-resource infrastructure actually supports it. While Agile may work in companies with specialized IT roles, it is definitely not designed and optimized for those scenarios.

OCP Open-Closed Principle

Another fundemental principle of object-oriented software design is the Open-Closed Principle. According to Uncle Bob (Robert Martin), this principle states that:

Software entities (classes, modules, functions, etc.) should be
open for extension, but closed for modification.

Essentially, what this means is that software's design should be such that it's behavior can be modified by extending the existing source code rather than modifying it. Consider the following example: public class CheckingAccount { private double _currentBalance; public void ProcessDeposit(double amount) { _currentBalance += amount; } } public class MortgageAccount { private double _currentBalance; public void ProcessPayment(double amount) { _currentBalance -= amount; } } public class Bank { public void ReceiveMoney(object account, double amount) { if (account.GetType() == typeof(CheckingAccount)) { CheckingAccount checkingAccount = (CheckingAccount)account; checkingAccount.ProcessDeposit(amount); } else if (account.GetType() == typeof(MortgageAccount)) { MortgageAccount mortgageAccount = (MortgageAccount)account; mortgageAccount.ProcessPayment(amount); } } } Notice that the ReceiveMoney() method of the Bank class has to determine the type of account passed in. Only based on that information it can perform the appropriate business logic. Later if we add another account type then we may need to modify this code in order for it to work properly. The above code is, hence, in violation of the Open-Closed principle. So how can we revise our code to conform to the Open-Closed Principle? Abstraction is the answer. Let's take a look: public abstract class Account { public abstract void Deposit(double amount); } public class CheckingAccount : Account { private double _currentBalance; public override void Deposit(double amount) { _currentBalance += amount; } } public class MortgageAccount : Account { private double _currentBalance; public override void Deposit(double amount) { _currentBalance -= amount; } } public class Bank { public void ReceiveMoney(Account account, double amount) { account.Deposit(amount); } } Notice how we got rid of all conditional logic in the ReceiveMoney() method. Also notice that if we later add a new account type, the ReceiveMoney() method will work without any modifications. The revised version of our code above, hence, conforms to the Open-Closed Principle. Let's now consider what happens to our code if a new business rule is added that requires an e-mail notification to be sent to the account holder whenever a deposit is made. Obviously, we'll have to modify the ReceiveMoney() method to perform that logic. Since we can't account for all possible scenarios, we can never acheive perfect closure. With that in mind, design the simplest software that will do the job, and conform to OCP or other such design principles where you see the need.

SRP: Single Responsibility Principle

SRP is one of the fundamental object oriented software design principles, first introduced by Tom DeMarco in 1979. In simple terms, SRP states that every class should have a single well defined goal, and that all members of that class (properties, methods, etc.) should work together to achieve that goal. Robert Martin puts it in slightly different terms:

There should never be more than one reason for a class to change.

Essentially SRP enforces the concept of high cohesion that should not only be applied to classes but also internals of a class, such as methods. So why is SRP such an important design principle? It is because a class with multiple responsibilities is more difficult to change and more difficult to reuse (design smells: fragility and immobility). While SRP is quite easy to understand, it's not always logical to implement. Consider the following class:

Given the convenience of having a customer class that is smart enough to know how to load and save itself, would you rather break it out like this?:

According to SRP, you should, because the combined customer class shares multiple responsibilities: customer business logic, and customer persistence logic. In fact if you use TDD, you may even be forced to separate these two responsibilities.

In practicality, however, while we should keep this principle in mind and try to apply it wherever it makes sense, we should never get so carried away that we start introducing other design smells in our architecture, such as Rigidity, Viscosity, and Needless Complexity.

Is Agile For Everyone?

Currently I am working at a company with a fast-paced environment, very aggressive deadlines, and historically a high rate of failure in the form of poor product quality and missed deadlines. Back when I started at the company, I introduced them to Agile methodologies and how they could benefit the organization in higher employee and stake-holder satisfaction, fewer or no missed deadlines, and better product quality. After some initial mixed reactions I was able to get them on board with the idea -- or so I thought!

While I have been successful in implementing many of the Agile practices such as earlier stakeholder involvement, EDUF, iterative development, code refactoring, unit testing, continuous integration, etc. in a relatively short amount of time, I still have to deal with the fact on a daily basis that they have to have very good estimates too far in the future. They must have this information from a budgetary standpoint and in order to be able to make executive decisions in advance about whether or not to proceed with a given project.

I have tried my best to explain how the Agile estimation works, and how trying to estimate too far in the future could be a wasted effort, but after all the time and energy spent, I am beginning to think that perhaps the Agile philosophy is not for all mindsets after all!