Knowledge is the New Gold – Happy Mining!

Do you use data to support your decisions? Most people use data to support even the simplest decisions on a daily basis. What route should I take to beat the traffic? Is the interest low enough to refinance my house? Should I go to this meeting? When should I wake up to make my 8:00 meeting?

Most business users evaluate data to support their decisions. In a typical scenario a business user has a hypothesis and explores data to evaluate his hypothesis. For example: a business user would advance a hypothesis that his company should receive a good return on investment (ROI) from direct marketing, based on the last year’s data of 50% ROI from the direct marketing.

Data Mining Benefits

In contrast, data mining helps to establish new hypotheses. The goal of data mining is to extract knowledge from large quantities of data and to expose previously unknown interesting patterns such as groups of data records, anomalies and dependencies. Ben Averch from eBay had a great quote on this in his TDWI presentation earlier this year: “The metrics you know are cheap. The metrics you don’t know are expensive – but high in potential ROI. “
If you shop online, you are most likely familiar with recommendations that an online seller displays on their sites: “Customers Who Bought This Item Also Bought…” and “Frequently Bought Together…” These recommendations are the result of data mining and predictive analytics.
According to International Data Corporation’s (IDC) 2011 research, the median return on investment for predictive analytics projects is 250%, which represents an increase from the 145% average ROI from IDC’s 2003 study.

Just a few examples where businesses use data mining solutions:

  • Gain new customers and reduce customer attrition
  • Minimize risk and detect fraud
  • Anticipate resource demands and future sales
  • Increase marketing campaign responses
  • Analyze customer’s profile and suggests products to purchase
  • Identify and resolve transportation bottlenecks
  • Pinpoint most effective law enforcement methods

… and many more

Generally data mining algorithms look for patterns and trends in data, based on the relationships between input columns (age, gender, location, profession, time, etc.), and the outcome column (e.g., the decision to purchase a particular product).

Data Mining with SQL Server 2012

If your business already owns SQL Server 2005/2008/2012 SQL Server Standard Edition or above, you have a number of data mining algorithms at your disposal as a part of SQL Server Analysis Services.

Simply install Data Mining Add-ins from http://www.microsoft.com/download/en/details.aspx?id=7294 and use cool reporting capabilities in Excel and Visio.

Microsoft provides helpful guidance on “choosing the right data mining algorithm”; here is a summary:

data-mining-diagram

  1. Discrete result has a few limited states, such as customer making yes or no decision about purchasing a product or a law firm making a decision to litigate a matter, based on win and loss probability.
  2. Continuous results have a range of states, such as seasonal trends or company’s future quarterly expenses.
  3. Sequence predicts customer’s navigation on a website: what is the probability of a customer clicking on a particular link. This can help a company to direct a customer to the fastest path to a sale.
  4. Groups of common items are the items that typically combine together. This can help to suggest additional products for purchase or design a packaged solution.
  5. Helps to find clusters of customers that exhibit similar behavior or identify personas for software development.

SQL Server makes it easy to switch from one data mining algorithm to another, to experiment with models and to discover the model that provides the most useful insights. You can leverage insights derived by one model to tune your inputs for another model. You can also combine results of data mining with On-Line Analytical Processing (OLAP).

In addition to the algorithms described earlier, SQL Server includes Text Mining algorithm that analyzes unstructured text data. Text Mining allows companies to analyze unstructured data such as a “comments” section on a customer satisfaction survey. Text mining algorithm is available in SQL Server Integration Services. While it is straight forward to create a package that would process text input with Text Mining algorithm, it is somewhat unfortunate that this algorithm does not follow the same implementation pattern as other data mining algorithms in SQL Server Analysis Services.

Discovering insight with data mining can be very rewarding. Knowledge is the new gold – happy mining!

Overcoming Obstacles in the Agile Transition

By Grant Beck, Agile Solutions Practice Director

Recently I was tasked with the challenge of converting two project teams to an Agile Scrum software delivery model. Our client’s management team had become more aware of the benefits of Agile and was anxious to put it into practice. I was somewhat surprised at the initial protests from a few individuals on the teams to working ‘Agile’. By definition, Agile means ‘quick and well-coordinated in movement’ or ‘marked by an ability to think quickly’. Who would be opposed to such a change?

Change is stressful and often unwelcome, especially when imposed upon longtime, established employees by an ‘outside’ consultant – we had significant potential for conflict! Success depends in part on salesmanship, part on political positioning, part on psychology, and in no small part on patience. The process we went through was both successful and rewarding, and I’d like to share a few a key points from my experience. While these points are in the context of transitioning to Agile Scrum, they can easily be generalized to organizational change overall.

Beware of “Best Practices”

One of the terms commonly used in business is ‘best practices’. Best practices are wonderful guidelines, but too often they are seen as the holy grail of the way we work. We rarely question a best practice. It is, after all, ‘best’! Therein lays the danger. Generally in business, and specifically in Agile, the desire is for continuous improvement. When we stop trying to get better we get left behind, we miss opportunity, and we stagnate. Best practices should continually evolve if they are to remain best practices.

Don’t codify the way things should always be done. Circumstances change, businesses change, and technologies change. Acknowledge the value that exists in current processes, but challenge yourself and the team to continually reassess. When you hear the term ‘best practice’, we always ask: “How can we improve our approach?” Best practices are general guidelines but they need to change based on the needs of a situation/project.

Think Big

Consider how the transition to Agile affects the whole business – not just IT. It is important to ‘socialize’ and sell this methodology to all levels of the business. A purely top-down or bottom-up approach will not be sufficient to enact a change. The functional team needs to understand their role in helping to define user stories. Management needs to understand the importance of persistently communicating down and reinforcing the expectation of the new work habits. Seek out individuals in all areas who are ‘on-board’ with the change and enlist them in perpetuating change throughout the organization. Show employees how the model works for them. Get people excited!

Be Willing to Adjust (it is Agile, after all)

Enacting change isn’t about cramming a methodology down someone’s throat. Don’t lose sight of the big picture – remember to pick your battles. It won’t do you any good to get into a heated debate about something like whether the team should use an online tool to track progress on a burn-down chart versus using a white board. Let that one go and keep the big picture in mind.

Negotiate. If someone on the team is insistent on using a certain document or form, be flexible enough to accept this if it means getting that person on board with the process.

Try enacting change in different ways. Maybe a big flashy PowerPoint demonstration about the benefits of Agile will help – maybe meeting with a team member one on one. Take some action and observe the response. Then try something different. Experiment with differing levels of pressure. Be creative and observant, adjusting your approach as necessary.

Be Respectful and Listen

I remember one meeting where we had introduced some of the Agile concepts to the delivery team and I knew there would be some very vocal protests about the changes we were hoping to implement. I also knew that this situation would likely turn to a ‘feeding frenzy’ of negativity and confrontation once the ball got rolling. I told myself before the meeting to expect this and to, under all circumstances, keep my mouth shut. It was important to ‘weather the storm’ and let the team express their frustration and ‘get it all out’ without my being defensive. It also allowed me the opportunity to assess each individual’s objections and begin thinking about how to continue the dialog once the fervor died down.

In Conclusion…

A common mistake made by consultants is to walk into a situation where they know of one way to implement a new methodology without right-sizing to a client’s needs, potentially ignoring existing processes or organizational structure. As consultants, we are guests in someone else’s house. Making an effort to listen to and empathize with the client will go a long way in establishing a productive dialog. As consultants, we may feel that our job is to come in and immediately apply our expertise to a situation when sometimes it is best to sit back and listen, discovering the best path forward.

Enacting organizational change and affecting the way a business works is difficult and challenging. I’ve learned a lot along the way and am pleased to see that change is possible with even with the most ingrained of processes and intransigent of clients. Plaster Group employees, by virtue of their caliber and experience, are well positioned to help customers adopt new and improved ways of doing things. The depth of our proficiency and breadth of our knowledge base allows us to leverage that experience to help our clients navigate to better processes and sounder solutions.

Plaster Group serving on the TDWI Northwest chapter board

The TDWI Northwest chapter has announced its board members for 2013 and Plaster Group is pleased to be represented by Wendy Parker (President) and Ted Schill (Sponsorship Coordinator). Look for exciting chapter events in 2013, including guest speakers Michael Scofield on January 22nd in Seattle (Data Visualization) and Bill Inmon on March 13th in Bellevue (Data Warehouse 2.0). You can learn more about TDWI Northwest on their chapter page or on LinkedIn.