Blog


How IT Service Teams Can Be Agile

Agile software development approaches were originally created to address software development challenges. As enterprises that don’t sell software as their main product adopt agile, they find the need to adjust their approach in order to apply agile across their IT organization and entire enterprise. 

In previous posts, I explored how you can use agile to work with COTS and business intelligence, both of which are common activities that are different from (but not entirely unlike) software development activities frequently found in IT organizations. In this post, I’d like to explore how you can perform another activity, IT service, in an agile fashion.

Characteristics of IT Service Work

IT service work includes areas such as service desk, release management and configuration management that are related to providing information technology services in an organization. There are several other areas that could be included in this description, and I could spark several semantic arguments about what is included in the IT service work category.

To keep things simple, what I’m talking about in this post are those teams in an IT organization that tend to have work with the following characteristics (as opposed to teams that work on software development projects):

Work items usually takes the form of a request or a ticket. These items show up at unpredictable times. Some items are extremely time-sensitive, especially if they deal with people not being able to get their work done, while other items represent changes which are not quite as time-sensitive. Each item is independent from every other item. This means that as soon as you finish the requested action you can deliver the results to the requester, without waiting to finish other, unrelated items. Items generally have less uncertainty surrounding them than items on a software development backlog might have. The only exception to this is when the item represents a bug that requires investigation to identify the root cause. Work Flows Through an IT Service Team

You can group agile frameworks into two groups: those based on timeboxed iterations, such as Scrum, and those based on flow, such as Kanban. 

Each type of framework is better suited for different situations. The context in which you want to apply each framework helps you determine which framework works better for your team to establish your methodology.

Flow frameworks are better suited to IT service work than timeboxed iterations. There’s a variety of reasons for that:

Flow approaches provide greater flexibility to deal with priorities that change frequently. You can change your priority every time you start working on a new item. Flow approaches are better suited for work items that show up randomly, are independent of each other and are generally not part of a bigger whole. Flow approaches are better suited for work items that you can deliver when you are done rather than those that need to be grouped together with other changes. How Work Flows in an IT Service Team

To describe how to apply a flow approach to an IT service team, let's look at a team that maintains a not-for-profit professional organization’s website, including membership administration, event registration and community groups.

Work flows into this team when someone submits a question, request or complaint via a “contact us” form on the organization’s website. The team can’t predict when these questions will come in, and the items vary in their urgency. The team also has a few ongoing efforts to introduce changes to the website driven by new programs or events.

The team has five members, each with their own area of responsibility. A few of the members have the ability to back up the others when necessary or deal with the more involved support situations.

The team is distributed so they use Trello to keep track of the items they must work on and the items that are in progress. Because the items do not all have a consistent workflow, they keep their board straightforward by dividing items into the following sections:

Status

Description

New

Newly identified items are placed here when either the team identifies work they need to do or someone submits a request via a form on the website.

On Deck

Items that have been triaged and have sufficient information to proceed.

In Process

Items that a team member has picked up to work on.

Waiting for Verification

An item where a team member has completed the work and has requested verification that the work was completed appropriately.

Verified

The requestor or other team member verified that the work was done satisfactorily.

Done

Change was implemented on the website or request was delivered to the requestor.


When someone submits a request via a form on organization’s website, the form sends an email notification which Trello converts to a work item in the “new” section. 

The team members take turns triaging the items in the “new” column. As part of their triage activity, they determine the urgency of the item and classify what type of work it is (i.e., does it relate to membership, event registration or website content). The team member doing the triage sets the appropriate label to indicate the type of work. They then move the item to “on deck” and place the new item in a specific order in the “on deck” section based on the item’s priority.

When a team member is able to move an item they were working on from “in process” to “waiting for verification,” they check the “on deck” section for a new work item to start on. The team has some rules that everyone uses to select an item that are based on priority and the type of work that each item represents.

Experiencing the Best of Both Approaches

Even though the team uses a flow approach, they’ve adopted some practices used by teams working in an iterative fashion, mainly because they found value in adopting those practices, not because anyone required them to.

A regular planning cadence

Once every other week, the team gets together to look at the “on deck” section to determine if the items are in the proper order, whether there are any new initiatives that they need to account for and whether there are any items in “on deck” that could be removed altogether from their board. 

This discussion is like a sprint planning meeting with the exception that the resulting queue - in this case the “on deck” section - is not set in stone. Essentially, this discussion serves to reorder the “on deck” items based on the team’s current understanding of the priorities.

Daily stand-ups at the board

The team gets together once per day to briefly discuss what everyone is doing that day and to determine if there are any in process items that have hit a roadblock. During these stand-ups, the team members discuss:

What the team needs to do to get things moved across the board. Based on what is known right now, what item the team should start working on next whenever there is room. Retrospectives around the board

The team also finds value once every other week to discuss how to improve their process. They have this retrospective while looking at the board, because it provides insight into how the team is doing. The questions the team asks during these retrospectives include:

Is there any hidden work that’s not represented on the board? Do we need to add a queue section? Are there any impediments? How can we remove them? Are we tracking things at the right level?

The team used these retrospectives to end up at their current process. They certainly didn’t start using a flow approach with the process described here. Rather, that process evolved as the team got some experience and held retrospectives to gradually improve the process.

Measure and learn

To aid with their retrospective, the team tracked a small set of metrics that also helped them to identify obstacles in their process. These metrics include:

Throughput: the number of items completed this week Lead time for each item (completed date and start date) Average lead time for this week Items completed with > 0 blocked days Total blocked days A list of places where items were blocked How Can You Get Started?

Would you like to begin using a flow approach in your IT service team? The best way to get started is to map your current process using a board similar to what I described previously.  Then, place all the work your team is currently working on in the appropriate section and start working together as a team to gradually improve your process using stand-ups, metrics and retrospectives.

Do you have any experience applying flow techniques in an IT service team? Share your experiences in the comments below.

Three Ways Agility Helps Business Intelligence

The ability to learn and adjust on an ongoing basis, a key benefit of agility, is extremely helpful for business intelligence and data warehouse initiatives.

If you’ve worked in business intelligence for any length of time, then you’ve experienced the ongoing discovery that happens with business intelligence efforts. Once you get a feel for the kind of information that business intelligence can provide, you come up with all kinds of things you can use the data for that you didn’t think of initially.

How can you experience the benefits of agility with business intelligence? In this post, I’d like to take a look at three ways you can use agility to improve the effectiveness of your business intelligence efforts.

Express Your Outcomes as Questions and Decisions

Every initiative is more effective if you can describe success in terms of the outcomes you want to reach instead of the outputs you want to deliver.

With business intelligence initiatives, those outcomes are best described as questions you want to answer or decisions you want to make. When you describe outcomes in that way, you will experience the following benefits:

You can deliver something quickly that provides your customers with value as you build out the rest of your data warehouse. When you go through a full pass of all the steps to deliver a data warehouse on a small subset of the overall data warehouse, you can learn some things that can be applied to the rest of the data warehouse. You can focus on only the data you need and avoid using data that will not be used, saving analysis, development and testing time. By organizing your work around specific outcomes, you can get a more useful indication of progress.

I’ve found that Socratic questioning is an effective way to uncover those questions and decisions. I recently wrote a post on BA Times that describes how you can elicit the questions users want to answer and the decisions they want to make.

You can capture these questions or decisions in the form of a user story (As a <who>, I want <what> so that <why>), job story (When <situation>, I want to <motivation> so that <outcome>) or other template of your choice. The format really doesn’t matter.

What does matter is that you identify distinct questions and decisions (outcomes) that are satisfied by big chunks of functionality you want to deliver (output). You can call the big chunks features, epics or whatever term you prefer--again, it doesn’t matter. I’ll use the term “big chunk” for the rest of this article.

By organizing your work around each big chunk, you can deliver the functionality that delivers the data formatted in a way necessary to answer a question or make a decision. If you focus on one big chunk at a time, deliver it to your customers and then get feedback on that big chunk, you can use that feedback to revise your approach to subsequent big chunks. You may decide to deliver different big chunks, the same big chunks in a different order or in another way altogether.

Whatever changes you make, you’ll find that you more effectively deliver the functionality necessary to answer questions and make decisions, and you can avoid unnecessary work.

Incrementally Deliver Value

A common assumption about business intelligence is that you must get all the data warehouse functionality to its final before it can provide any value. However, you may find that you can provide value in many ways before getting to the final desired form of data access.

Even though your user ultimately wants to get to a daily report with all the data they need, you may find you can manually extract specific data from the source systems and place it in a single table that your users can manually query to get the answers to their question.

Once you verify that you have the correct data, you can automate the process, organize the data in a more user-friendly manner and perform necessary transformations to the data as you extract it from the original source systems and provide it to your users.

In his book “Agile Data Warehousing,” Ralph Hughes provides a way to slice up the big chunks of business intelligence work into smaller improvements to the process as described above. There are a variety of characteristics of both the user-facing data view and the back-end data storage that you can gradually improve to achieve a level of functionality that provides the most value for your users.

Here are a few of Hughes’ suggestions: 

Data View (Front End) Decomposition

Refresh Frequency

How frequently is the user’s data view updated?

●     Daily

●     Weekly

●     Monthly

●     Quarterly

●     Yearly

User Friendliness

How can the user access the data?

●     Single Table Access

●     Mobilized access (linked tables)

●     Defined navigation

●     Picklist supported

●     Dashboards

Automation Level

What triggers a refresh of the data view?

●     On demand at workstation

●     On demand posted to server

●     Scheduled refresh on server

Transformation Type

What transformations (if any) happen between data storage and the data view?

●     Direct data transfers

●     Applied business rules

●     Aggregation

 

Data Storage (Back End) Decomposition

Refresh Frequency

How frequently is data from the original sources updated?

●     Daily

●     Weekly

●     Monthly

●     Quarterly

●     Yearly

Refresh Type

When a refresh from the original sources occurs, how is the data updated?

●     Direct Link

●     Snapshots

●     Error recycling

●     Schedule driven

●     Manually invoked

●     Incremental loads

Transformation Type

What transformations (if any) happen between the original sources and the data storage?

●     Direct data transfers

●     Applied business rules

●     Aggregation

Target Layer

What layers in the architecture are updated when a data refresh occurs?

●     Staging

●     Integration

●     Linking history tables

●     Linking tables

●     Fundamental tables

●     Reference tables

●     Presentation

●     Fact tables

●     Historical dimension tables

●     Non-historical dimension tables

●     End user access views


The idea is to gradually improve on a specific characteristic one small chunk at a time. For example, you may start off with users, manually querying directly against the data storage using SQL. You can then improve this experience with small chunks that introduce each of the following changes:

Offer access through a controlled querying environment where links between tables are pre-built Provide data in a report that the user can refresh manually Automatically deliver the report to the user on a regular schedule

After each step, get feedback from the users to determine if you have met their needs or if additional improvements are necessary.

Adopt Different Technical Practices

To effectively deliver data warehouse functionality in an incremental fashion, you’re going to need to adopt some different technical practices than what you may be accustomed to from your past experience working on business intelligence initiatives.

First, it’s important to remember that you will change your database structures throughout the process of building the data warehouse, and that’s okay. To make that process as smooth as possible, it’s best to create Data Definition Language (DDL) scripts to make changes to the data structures as well as Data Manipulation Language (DML) scripts to update the data in those structures. Then, include those scripts in the version control along with the processes you use to run your ETL processes.

In this way, you have clear means of moving from one version of your data warehouse to another and you have a straightforward way to automate that process.

Another helpful practice is to create tests to include with the DDL and DML scripts to confirm that the changes to the database structures and the initial loads of data are performed properly.

You may also find it helpful to extend a test-driven approach to all the code you write for the data warehouse, including not only the scripts to make changes to the database structures, but also the processes you use to update the data warehouse on an ongoing basis.

Ken W. Collier’s book “Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing” contains a great deal of information on how to apply techniques such as Test-Driven Development to a data warehousing environment.

Agile Business Intelligence Helps You Learn

The main advantage of using an agile approach with your data warehouse is the ability to constantly refine your understanding of what your customers need and to make sure you spend your time on valuable work, delivering only the data that your customers need at the frequency they need it, organized in a way that is most useful for them.

Do you have experience approaching business intelligence in an agile fashion? Leave your experiences or questions in the comments section below.

How to Implement COTS in an Agile Fashion

Most organizations do not develop 100 percent of the software that they use, nor should they.

A vast majority of the jobs they have to do can be satisfied with readily available Commercial Off the Shelf (COTS) software, or its more modern successor, Software as a Service (SaaS).

Why go through the pain of building something to address the same need that thousands of others have and that many developers have put in the effort to figure out already?

Yet, the organizations that primarily use software developed by someone else are the same ones that are now trying to become more agile, at least in their IT departments. Those folks look at the various frameworks for agile software development (primarily Scrum and SAFe) and say: “That’s great that you’re building software, but what if you primarily purchase and implement it instead?”

Here are some ways I’ve found to implement COTS and SaaS in an agile fashion.

The Goal Is to Get a Job Done, Not To “Be Agile”

I thought I’d get this one out of the way early on.

Being agile should not be your ultimate goal. Rather, your ultimate goal should be to deliver the right outcomes, which you can think of as satisfying your customer’s needs. Agile values provide a great mindset to have in place to accomplish this, but they are simply a means to an end.

Agile frameworks (such as Scrum) provide a starting point for your team to figure out how you’re going to approach work in a way that embodies the agile values and principles which will most likely increase your effectiveness in delivering the right outcome.

In the case of COTS and SaaS, you have a job that you want to get done that is similar to a job that many organizations have, and there are existing products that will help you get that job done more effectively than building something yourself.

Instead of asking how you can implement a COTS software in sprints, ask how you can implement this package in a way that delivers the most value for your organization and allows you to address the uncertainties you run into along the way.

Fall in Love With the Problem, Not the Solution

All too often, organizations implement COTS or SaaS products because someone with a great deal of influence or authority came across a “must-have tool” and began hunting for a reason to introduce it to the organization.

A better way to approach this is the from the opposite direction: start with the problem in mind and describe that problem in the terms of what value it has for the organization. I find that collaborating with impacted stakeholders to create a problem statement can help you build a shared understanding of the problem.

When you understand the issue at hand and want to identify possible solutions (some of which may not actually require software), you may find a technique such as Impact Mapping helpful.

Once you’ve identified desirable characteristics of your solution (hint: do not just look at the feature list of someone’s pet product and crib off of that), describe those characteristics in the form of user stories or job stories so that you can focus on the problems you are trying to solve. 

Pick the Right Solution

Now that you understand the problem and know what you’d like in a solution, you can decide whether it makes more sense to build or to buy. Using purpose based alignment can help you determine the right path. If your solution will differentiate you in your market, it probably needs to be unique, so building the solution is most likely the better choice.

If the solution is parity for your market, you may find that you can buy an existing solution.  Should you go this route, resist the urge to customize whatever it is that you buy. That is a sure way to blow your budget and timeline and probably fail at achieving your desired outcome. 

Instead, find the solution that is the most closely aligned with your desired characteristics, and revise processes or expectations in the small cases where you don’t have alignment. This is a parity activity. There is no value for you to do it more uniquely than anyone else.

Iterate and Increment to Learn

When most people think of applying agile to COTS or SaaS, they are trying to figure out how to fit the work of implementing the solution into sprints. There is certainly validity into figuring that out, but it’s essential to understand why.

You don’t iterate and increment for the sake of doing things in sprints. Rather, you iterate and increment in order to learn.

Most COTS and SaaS solutions require some amount of configuration, integration with existing systems and the transformation and loading of data from a current system. These activities provide great opportunities for an iterative and incremental approach. The two primary ways you can approach it are:

Gradually roll out functionality. Start using the new solution for limited purposes. Perform the configuration and integration to support just that functionality, deploy it and observe the results. Determine what you can learn from that rollout to apply to, configure and deploy subsequent functionality. Gradually roll out to new users. Have a small subset of the total planned user base start using the solution and closely watch the challenges they run into and the feedback they have. Use that feedback to alter your configuration and approach to roll out to the next set of users. When you pick you first users, make sure you select people who are comfortable with change and adept at using products so that they can provide meaningful feedback.

You may find that in some situations you can combine both approaches, especially when different groups of users use different functionality. The key to picking the right approach for you is to select the one that will allow you to learn the most with the least amount of impact to ongoing operations as you make your transition.

And, if you must transition data or content from your existing solution to a new solution, do not wait until the last minute to transition that data over. Determine how you can transition it as soon as possible. If the data changes rapidly, you may have to wait until just before go-live time to transition that data over. If it rarely changes, move it over immediately so you can test your new solution in as realistic a setting as possible.

Agile COTS/SaaS Is About Value and Learning

When you implement a COTS or SaaS solution, figure out the problem you’re trying to solve, have a clear understanding of how the solution will help you solve that problem and implement it in a way that will help you learn along the way. Doing so will help you deliver the right outcome to your organization.

What experiences have you had implementing COTS and SaaS solutions? Did I miss anything? Share your thoughts in the comments section below.