Posted on in High Performance Data Teams

Tips for Managing Priorities in a Data Team

We work with a lot of different Data Teams, and most of them are faced with the same challenge:

How do you handle all of these competing requests for information?

Below are some relatively easy-to-implement tips for dealing with this situation, but first let’s see why this can be so hard. The following are some of the more common reasons we’ve seen in the field:

  • Every request seems to be urgent. Most Data Teams are all too familiar with the expression “we need it yesterday”.
  • Every request seems to be very important. How can a Data Team not give priority to a request that comes from the CEO’s office or from the Board? What about situations where Public Relations needs good information to handle an emerging PR issue?
  • Requests for information are “free”, meaning that in most situations, the people requesting the information don’t have to pay for it. As a result, demand for information grows much faster than the capacity of the Data Team.

Overloaded Inbox

Here are some tips for Managing Priorities in a Data Team:

1) Keep a log of all active requests
As simple as it sounds, keeping an up-to-date log of all active requests is a “must have” enabler for managing competing requests in a Data Team. Many Data Team leads feel that they don’t need such a log, citing that they have it all under control, and that they are too busy to keep another list up to date. But such a log can help identify the capacity needed in the Data Team, and the skill mix that’s required. At minimum the Active Request Log should include the following information for each information request:

  • Who is asking for the information?
  • What are they asking for?
  • When did they ask for it?
  • Who in the Data Team is handling the request?
  • When did we promise to get it done?
  • What’s the status of the request (not started, active, completed, cancelled)?

In addition, the following information can be very helpful for planning purposes:

  • When was the information delivered?
  • How many hours of effort were involved in preparing it?
  • Was the due date pushed back? If so, how many times and by how many days?
  • Was there any feedback from person who requested the information?

This list can be as simple as a whiteboard, a shared spreadsheet, a SharePoint list, or a Google Doc. The hard part is having the discipline to keep it up to date.

2) Review the log as a Data Team every day
Having a daily 5 minute meeting as a Data Team may seem like a big burden. Who needs another meeting in their already-too-busy schedule? But if done right, a daily 5 minute meeting to review the Active Request Log can help a too-busy Data Team work together to make sure that the most important things are being worked on every day. Specific things that can be clarified during this 5 minute check-in include:

  • What must we get done today?
  • What must we get done in the next couple of days?
  • Who has the lead on each piece of work?
  • What requests need more support?
  • What counts as “good enough” for the requests that we’ll be working on today and tomorrow?

This quick meeting can set the entire Data Team in the right direction at the start of each day, and in doing so, go a long way to reducing the last-minute scramble, and make sure that the Data Team works to it’s full potential as a team.

3) When handling new requests, use the active request log to set expectations
If you have the discipline to do the above 2 steps, then after not too long you will have great information for managing expectations with new requests. For example, if there is a last minute urgent and important request for information, then at minimum you will now know:

  • How long will this really take us to complete?
  • Are there any recent requests for information that are similar to this one? If so, can that requests be modified to meet this urgent need?
  • Will any active requests not be completed on time, as a result of this new urgent request? If so, is the person making this new urgent request willing to take the heat?

In a lot of respects, most Data Teams are carrying out all of these three functions, but often it’s in people’s heads. By adding a little bit of tracking and daily discipline, the Data Team can significantly improve their work effectiveness, and at the same time better meet the needs of their customers.

We’re sure you have perspectives of your own on this subject. If you so, please share your thoughts and ideas.

Note: What is a Data Team?
When we refer to “Data Teams” it’s a catch all for groups of technical, statistical, and subject-matter domain experts that are involved in providing information to support their organization. These teams are sometimes called “Business Intelligence”, “Decision Support”, or “Information Management”, but they can also be internal consultants such as “Operations Analysts”, “Strategic Information” or “Research”. Many of these concepts equally apply to teams of Data Scientists.


Posted on in Driving Action with Analytics

Applying “Purposeful Abandonment” to Big Data

I’ve recently been reading “Inside Drucker’s Brain” by Jeffrey Krames. I’ve read some of Drucker’s hits, but I found this book put his great ideas all together in an easy to digest format.

One of the Drucker concepts that resonated with me is the concept of “purposeful abandonment”. He argues that it’s easy to take on more responsibility, do more products, support more customers, but the hard part is the “letting go” part. By taking a concerted and proactive approach to identifying “what you won’t do anymore” one creates the space needed to move forward in the areas that matter.

The concept is surprising relevant when applied to Data Science. Here’s my take on it:

1) Do you really need all those data fields and metrics?
The thrill of Big Data is having no limits on the number of fields that we have in our datasets. With space being so cheap, and an abundance of distributed computing power, there’s no need to scrutinize the fields that we’re tracking. But, isn’t this just a form of Parkinson’s law in action (i.e. Data expands to fill the space available for storage)? With every data field and metric comes the need to do quality assurance, test for face-validity, and understand the underlying quirks. Letting go of those “nice to have” data fields and metrics allows Data Scientists to better focus on the ones that really matter. Less time checking redundant fields and metrics equals more time for insightful and impactful analyses.

Saying No

2) Do you really need all those records?
Just like the previous concept, what’s the big deal? Why not analyze all the data records in our data sets, all the time? There are certainly times when we really need the full dataset, but often this stage can wait until the first exploratory analyses have been done. Sadly, some analysts can get stuck in a mindset of always running analyses on the full dataset. And so, they spend lots of time and effort on using Big Data tools, when they could have used good old fashion statistical samples to just cut to the chase. Less time running all analyses on all of the data records can equal more time nimbly running exploratory analysis to find the hidden gems you’re looking for.
(more…)

Posted on in High Performance Data Teams

New Year’s Resolutions for Data Scientists

As a group, Data Scientists seem like the type of a people that would seize any opportunity to improve. So in the spirit of fun, the following are 4 “tongue in cheek” resolutions for this year.

1) Gain More Weight
Data Scientists are getting a lot of attention these days, which is great. We need to continue to gain our collective weight as people who help other people make sense of the ever-growing mass of data, translating what the numbers mean into something actionable for non-Data Scientists.

Data scientist

2) Keep Smoking!
Yes, really, keep smoking! The concept of the Data Scientist is smoking hot, and in a self-promotion kind of way, it makes sense to keep this momentum going. So this means doing things like being a good ambassador of Data Scientists as a group, and explaining to people (i.e. your mother, your neighbor, the person on the street) what the heck we do.

3) Learn a New Language … Spanish, SQL, R …
Data Scientists are human too, and so it’s not uncommon for a Data Scientist to get really comfortable with a set of analytical tools – almost too comfortable. This could be the year to broaden your horizons and try something new. Different technologies often have completely different ways of approaching the same problem, and some are better than others depending on the task at hand. Knowing the options can save a lot of time in the long run. The article Top Holiday Gifts for Data Scientists has some good references for books and other resources.

4) Learn How to Make Friends and Influence People
Data Scientists can suffer from being too analytical, too technical and just too darn scientific. The greatest insights in the world don’t matter if they can’t be communicated to people in way that they can be understood. Data Scientists often can do with a little help in this area. These are two books that I’d recommend for Data Scientists that are looking to improve their game at presenting:

And let’s not forget the “making friends” part. The Data Scientist community is a growing one, and as good friends there’s a lot we can learn from each other.

I’m sure there are more resolutions in store for Data Scientists – please share your suggestions and thoughts.


Posted on in Driving Action with Analytics

How to Get Started With Simulation

Many business analysts decide that they want to start using simulation not just because it’s flashy and high-brow, but also out of pure necessity. These business analysts have taken their spreadsheets as far as they can and are at a point where the spreadsheets are becoming unwieldy and ineffective at providing reliable answers to their important business questions.

1 2 3 4

These analysts often ask “How do I get started with simulation?” Is there a course that one can take? Is there a tutorial? Is there a book? Ultimately what’s the best way to get started? Here are 4 questions that I suggest they consider:

Question 1: Are you sure Simulation is for you?

I have a belief that most people can learn most things if they are motivated enough, and I believe the same is true with simulation. However, there are some skills that make the learning curve easier:

  • Are you logical and process oriented? The guts that drive simulation models are driven by process logic. If you’re able to look at a real-life business process and convert it into a meaningful and clear process flow map then this is a good thing.
  • Have you done any programming? There is a lot of “If … then” logic in simulation models, and having experience with programming (including VBA and complex spreadsheet logic) will only work in your favor. Simulation models are almost never programmed correctly the first time, so debugging skills are also very important.
  • Are you good at handling a lot of data? There is a lot of data handling to estimate inputs for simulation models, and most simulation model will generate a mass of data. This is a very important skill in being an effective simulation modeler.
  • Are you good at experimentation? Simulation is like a sand box, and experimenting with your model is a key part of developing, calibrating, and validating your models, as well as designing and carrying out scenario analysis.
  • Can you work without perfect information? It’s a routine experience that simulation models will need parameters and factors that have no available data. The simulation modeler often needs to form credible assumptions as a workaround to having to deal with incomplete information.

If you can answer “yes” to the above questions then simulation might be a good tool for you.

Question 2: Are you just dabbling or are you ready for a deep dive?

Simulation is often described as both an art and a science. Simulation is one of those skills that seems to be better developed through “doing” rather than reading books or taking courses. I’d highly recommend taking courses if you’re convinced that simulation is for you (I taught a simulation course for 5 years at the
University of British Columbia). However, what you learn from a course won’t really stick unless you are going to be able to work on a real simulation project shortly afterwards.

Simulation is one of those skills where it’s difficult to be effective until you’ve been working with it for a while (i.e. your second simulation project will be much better and easier than your first,… your third simulation project will be even better, and so on). If this is something that you’re just looking to add to your resume then I wouldn’t bother. People who hire talent for their simulation skills can easily differentiate between a dabbler versus an experienced practitioner.

A “deep dive” simulation project allows a new simulation modeler to really understand what they can and can’t do with a simulation model, the effort involved with various model elements, and the associated value that those elements add to the final conclusions. New simulation modelers sometimes learn through their first intense simulation project that simulation isn’t really for them.

Coloring numbers

Question 3: Do you have a “Simulation Worthy” problem?

Simulation is an invaluable tool if it’s applied to an important problem that cannot be solved using traditional tools. But, if you could effectively answer the same problem using a spreadsheet, then why wouldn’t you just use a spreadsheet? If the business problem isn’t important enough to justify spending days (sometimes weeks) of effort programming and validating your model, then you could be creating a situation where your organization perceives simulation modeling as high effort for low value.

Ideally, you would be in a situation where there is a business problem that has high value (i.e. the potential to support a million-dollar decision, or a strong potential to reduce risk, or increase efficiency). And, ideally, the problem involves complex inter-relationships between resources, and/or processes – the type of logic that is very hard or impossible to set up in a spreadsheet. And finally, the situation requires a handling of uncertainty and variability in order to fully address the business problem. We would argue that if you don’t have a problem that is “simulation worthy” then it’s best to wait until you do.

Question 4: Do you have a good example to start from?

Simulation is not like other mathematical and theoretical disciplines, in that there is no single “right answer”. There are many different ways of modeling a system, all of which can be valid (provided that the assumptions are disclosed), and it often comes down to a balancing act of model accuracy versus model complexity. Simulation modelers often add more detail and logic into their models in an effort to improve the accuracy of the model, but as they do so, the model typically becomes more complicated, more difficult to debug, more difficult to validate, and more difficult to run scenario analysis on.

When new simulation modelers are getting started it can be difficult to make these decisions. A great way to learn is to partner up with a mentor – ideally someone who has done a few simulation projects where the results actually supported a decision outcome. The INFORMS Simulation Society is a good place to start (), and if you can do it, attend the annual Winter Simulation Conference next November.

If you can’t find a mentor, try learning from example models. Our company AnalysisWorks, made a simple simulation model of an Emergency Department that is 100% free and available for
download. Without any programming, you can interact with this 3D animated model to get a sense of the types of things you can do with a simulation model.

Simulation Model




Posted on in High Performance Data Teams

The Science of Data Scientists

The concept of the Data Scientist may very well be the next big thing in the field of analytics. Recently several industry leaders have weighed in on the question “What is a Data Scientist?”, but another way of looking at this is to ask the question “What is the Science of Data Scientists?”
            Data Scientist

A dictionary definition of science is a “systematic knowledge of the physical or material world gained through observation and experimentation”. So let’s look at the use of science in three areas that Data Scientists all need to do in carrying out their basic work:

  1. They transform the data into a format and structure that is conducive to analysis
  2. They carry out some kind of descriptive, interpretative, or predictive analysis
  3. They communicate their results

Using Science in Data Transformation:

Anyone who’s worked with data for a while knows that the data you have available is usually less than perfect. Missing data, inconsistently formatted data, and duplicate data are fairly routine obstacles, and then linking data from different sources is even more challenging. Data Scientists are also often required to work with “secondary data” that has been generated through an operational system or process. The data was originally designed to meet a functional requirement, rather than with the intention of it being analysed in the future. Even if the data is clean and error-free, there is a requirement to reorganize the data into a structure that is conducive to the analysis that needs to be performed.

So, in response, most Data Scientists develop skills in transforming data, and are quite good at it too. They use tools ranging from statistical analysis software to standard database technologies. Where the science comes in, is that there if often a lot of experimentation that takes place along the way, as the Data Scientist figures out how best transform the data while introducing little to no error along the way.

Many Data Scientists have learned the hard way that using a scientific method to prove that the data transformation has been done correctly ultimately saves time and reduces rework in the end.

            
Big Number

Using Science in Performing Analysis:

Here the use of scientific method is more obvious. It is taken as a given that Data Scientists conduct their analysis and modeling systematically, and that the essence of the work involves observation and experimentation. In carrying out the work, often “the proving” is a key component of what the Data Scientist does, so that they know they are drawing the right conclusions.

However, there is a wide range of scientific tools that Data Scientists can use to understand and interpret massive amounts of complex data. Data Scientists are not unlike other skilled experts, and can be sometimes be like a carpenter with a hammer who sees every problem as a nail. For example, some Data Scientists are truly exceptional when it comes to logistical regression modeling (making the best guess of a “yes/no” variable), but then are complete novices when it comes to multivariate analysis (such as condensing information captured in 1,000 correlated variables into 10 summary variables). As is often the case with niche skills, it takes a while to really get good at using them effectively, and it’s rare to find Data Scientists that are truly effective in all domains. The scientific connection here is that Data Scientists sometimes have to come to grips with the limits of their own skill set, and have to experiment in new directions to expand their knowledge base.

Using Science in Communicating Results:

This angle is less intuitive, but ultimately what’s the point of doing high-brow analysis, if nobody is able to understand the result, or even worse, if they can’t use the result to support a key decision?

Data Scientists that are in high demand are those that are able to truly understand the business question being asked, and why it’s being asked. Then they communicate their complex findings in a way that the decision-makers can actually do something with the result.

This important skill takes a while to develop, often through experimentation (i.e. what happens when I present it this way?), and then observation (i.e. what did the CFO do with the last findings I sent her?). Even better, is when the Data Scientist adopts basic market research approaches to their own work. Specifically, by following up with their clients and/or end-users of their work and discovering how the results could be even more useful. Or taking a more traditional approach, they can literally post their results with on-line reporting tools and run analytics to see how often and how deeply their results are being viewed.

The concept of the Data Scientist is still relatively new and will be shaped by those of us who work in and around in the industry. Please offer your own comments and feedback, even if you disagree with any of these ideas.

Posted on in Driving Action with Analytics

How to Allocate Resources as an Executive Team

Many executive teams feel that they could improve how they make decisions about resource allocation. These are decisions such as “which strategic initiatives should we approve for this year?” or “how much budget should we allocate to marketing versus customer service?” or “how many beds should be allocated to the surgical program, versus the medical program?” And this is at a time when organizations are all trying to do more with less – more sales with less sales staff, more shipments with less cost per shipping, more strategic initiatives with fewer leaders to push them forward, and so on.

The following are some of the common challenges that executive teams face when they are making decisions about allocating resources as a team:

  • Different people involved in the decision-making process have different goals and objectives. A win for one participant is a loss for another.
  • Decisions are often influenced by personality and emotion, as opposed to based on evidence.
  • Decision making processes have no feedback loop. As a result nobody keeps score on the quality of the decisions, and decision-making doesn’t improve over time.

    • Decision making

    Some teams have frameworks that they use to support decision-making, but it’s not working for them because the process, framework, and technology (i.e. spreadsheet, decision-support system) is too complicated to use, or it’s too cumbersome to maintain.

    Even worse, is when the team is attempting to use an “off the shelf” solution that doesn’t do a good job of capturing what’s important to them as a team.

    At AnalysisWorks we’ve successfully developed and implemented solutions that help teams make resource allocation decisions. These solutions have been developed over years, and we’ve learned a lot of painful lessons along the way. Here are 5 tips for success for allocating resources as an executive team:

    Tip 1: See resource allocation as a decision-making process …

    … as opposed to a one-time event. For example, it may very well be that you only decide which IT projects will be approved once a year, however, a year goes by quickly, and you’ll be right back at the decision making point soon. See the design of an effective process as an investment for your future, and a means to make the most out of your scarce resources.

    Tip 2: Define the common goals that your allocation should be based upon.

    This should be something that the entire team can get behind. The financial side is often over-represented in these types of decisions, so it’s important to round out the goals to include non-financial aspects as well. Ideally you should see a connection here to your mission statement, organizational values, this year’s strategies, and your overall strategic plan.

    Tip 3: Decide on the rules of the game.

    Again, the entire executive team should agree on the rules of the game at the very beginning. The rules should be fair and transparent. The executive team should avoid reverse-engineering the process to justify one-time decisions, and instead the process should be based on agreed-upon principles. Ultimately, if the rules of the game are set up right, they will serve to communicate to the entire team what behaviours will be rewarded. It’s at this stage where the team will need to agree on what objective inputs will be input into the decision making process.

    Tip 4: Have an objective party “keep score”.

    Building on the previous tip, it’s important that the rules of the game have accurate measurement, and that the score keeping is fair. This is a situation where an objective group or individual should be at the center of the process making sure that inputs to the process are accurate and consistent. Even more important, this person or group will have the difficult job of overseeing that the outcomes of the resource allocation are measured. Specifically, they will be instrumental in closing the loop with respect to the question “Did we get the outcomes we were expecting from our resource allocation decisions?”

    Tip 5: Improve now, keep it simple, and learn as you go.

    Sometimes executive teams try to make the “perfect” decision-making process before they are willing to use it. At the end of the day, making a minor improvement over your current state is still an improvement in the right direction. For example, if your executive team makes resource allocation decisions based on purely financial information supplemented with qualitative information, then an improvement may be to quantify a non-financial consideration (i.e. Scoring “alignment with our three year strategy” on a High, Medium, Low scale is better than nothing). These processes can grow out of control, so it’s important to keep it as simple as you can get away with. And finally, chances are that along the way the executive team will identify how the resource allocation process can be improved next time. These improvements can and should be incorporated into the next round of decision-making to make the best, simplest process possible. You will know that it’s working if your executive team feels comfortable with the process, and is able to support the resource allocation decisions that are made.

Posted on in Driving Action with Analytics

What Every Executive Should Know About Simulation

Simulation modeling is an emerging management tool to support big decisions involving complex operations.  The technology has been around for decades and in the last while it’s become increasingly easy to apply to almost any operation, such as patient flow in an Emergency Department, bits of data through a telecommunications network, or a global supply chain.

Just to be clear, the kinds of simulations we’re talking about here are of operations and/or system flows, not flight simulations or video games. 

Patient Flow Simulation

Simulation is not for every organization, but it can be a valuable tool to have in your arsenal.  Unfortunately, though, most executives aren’t even aware that it exists.  The following tips meant to introduce the topic of simulation are based on 20 years collective experience using simulation in both consulting and academic settings.

Tip 1: Simulation can be a powerful tool to look before you leap

Having a simulation of your operations can be a valuable asset for providing insight into the potential impact of crucial decisions.  They can provide powerful supporting information to aid decision-making and investigating the associated costs and benefits of multiple decision options.

Simulation models can be very useful for big decisions: Situations involving the buying, selling, or reconfiguring or key system resources are ideal for simulation analysis.  They can allow organizations to analyze multiple potential scenarios and provide insight into nearly any situation involving uncertainty.

Big risk/reward industries such as finance, oil and gas, aerospace have used it for years, but now simulation has become very accessible.  Think of a simulation model as the next step up from your most complex spreadsheet.

Tip 2: Don’t use simulation if you’re not ready

Simulation can be expensive between the costs of staff time and software licensing so it is important assess whether or not your organization has the pieces in place to successfully develop a compelling model.  It may be premature to jump into a building a simulation if you don’t have a good profile of your activity, work flows, and financials.

Typically, organizations that use simulation effectively have already answered as many questions as they can with spreadsheet analysis.  It’s better to use simulation to address business questions that can’t be addressed effectively with spreadsheets (i.e. situations involving complex logic, multiple flows of activity, uncertainty),  rather than attempting to use simulation on a problem that doesn’t need it.Simulation Logic

Tip 3: It’s not what software you use that’s important

Many tools are available on the market ranging from simple spreadsheet model to complex custom software, but more important than the tool you are using, is the team running the tool.

You need good talent to make a good simulation: There is a big difference between a good programmer and an effective simulation analyst.  An effective simulation analyst understands:

  • How to keep the model manageable, flexible and scalable
  • The importance of validating the model so you know the results can be trusted
  • How to come up with good ideas for different scenarios

It is an important first step to decide if your organization is interested in getting into simulation or not.  If you’re unsure, you could try working with a university or a consulting firm to try it out.  If you are committed to getting into simulation, a critical first step should be to focus on recruiting and developing appropriate talent.  It takes time to get really good so don’t train someone if they won’t have an opportunity to use it intensely at least a few times a year.

 Tip 4: Think of it as a “sand box” rather than an optimizer

It is important to set expectations right from the start.  Simulation is not a crystal ball that shows you the solution to all your future problems.  It will not tell you what to do in a given situation.

It is, however, a tool for creatively testing different potential ways of running you system.  It will tell you that if you run your system in a particular way, then this is what you may expect as a result.

The ability to generate and interpret meaningful operational scenarios is why having an effective simulation analyst can be the make or break of a successful simulation modeling project.Aerospace manufacturing simulation

Tip 5: You should consider using simulation if …

So how can you tell if simulation is the right tool for you?  The following are some guidelines to let you know if your situation could benefit from using simulation modeling:

  • You have a big decision to make with high potential for risk or reward.
  • You cannot afford to make mistakes and it’s worth investing the time and effort to make sure new processes work as good as they can before being implemented.
  • You have a good understanding of your operations and system data.
  • You are able to do your first project with an analyst or team that has a track record of successfully using simulation
  • You are innovative and ready to utilize a new management tool.
  • You are committed to use the findings and recommendations, even if they tell you what you don’t want to hear.  After all, there is no point in wasting your investment.

 

Posted on in Driving Action with Analytics

Escaping Key Performance Indicator Hell

Buchenwald-100625-14486-Schwerte-hell
Image via Wikipedia

Key Performance Indicators (KPIs) are all the rage these days.  Many leaders of industry have instant access to their KPIs, but still in their gut they know that there’s much missing from the dashboard.  

The famous Peter Drucker quote “What gets measured, gets managed” rings true here too.  A bad set of KPIs can result in a lot of unproductive busy work, or as I like to call it Key Performance Indicator Hell. 

Hopefully that’s not your situation, but if it is, here are 5 tips for to success that can help you drive performance through KPIs.

Tip 1: Emphasize the word ‘KEY’ in Key Performance Indicators

The reporting tools that have made it easier to present all your KPIs in a million different ways, created a flurry of new problems:

  • Too many KPIs:  By the time you’re reporting on more than 10 KPIs, chances are that some will look good and some will look bad, just due to normal variation.  What does it mean to be doing great in 5 KPIs, ok in 3 KPIs, and terrible in 2 KPIs? 
  • Too much overlap in KPIs:  Often dashboard reports show signs of major scope creep as the report writer attempts to please all viewers.  When dashboard reports contain 2 or more KPIs that essentially represent the same underlying concept (i.e. total widget units shipped, total boxes of widgets shipped, total widgets weight shipped, etc) it makes it distracting for viewers to know what KPIs they are supposed to focus on.
  • Too many options:  Just because it’s easy to report KPIs in a variety of ways, it doesn’t mean that it’s the right thing to do.  (more…)