The Invisible Man

We have this problem at work.

Each of our clients have a queue of sales agents. These agents work at different locations and on different teams and are responsible for different kinds of leads. So for example, Bob at the Portland store is responsible for "special finance" leads, while Mary at the Gresham store is responsible for "third-party" leads. And of course a given team at a given location can have multiple agents working those leads and they can be distributed in a "round-robin" fashion where the leads are divided evenly amongst the available agents, or they can be given specific distributions, i.e. 70% go to Bob, 30% go to Mike.

Sometimes the agents will enter these sales leads into the system manually. Other times (and possibly even more often depending on the store and the agent) these leads are entered through an automated process which imports them daily on a schedule or they're entered through some action the user takes on the client's website. To further complicate things, some of these leads come to the client from various "third parties" (who shall remain nameless) and the client can individually configure the third-party provider so that its leads are assigned to someone other than the third-party team, like the special finance team.

Agents individually can be assigned a maximum number of leads, so if none of the agents on the appropriate team can receive any more leads, then there can be a "catch-all" agent to which leads for that team are assigned when no one else is available. If there isn't a catch-all agent for the team, then there can also be a catch-all agent for all of the client's teams.

None of which begins to touch on the fact that if a new lead appears to be from an existing customer, it is supposed to be assigned to the agent who handled their most recent previous lead.

Are you confused yet? You should be.

These sorts of complex decision trees (that really should be flow-charted if you wanted to see what's going on), are the sort of reason we started creating business software in the first place.

So what happens?

The client sets up their teams. They assign Bob and Mike to the third-party team and get to work. Everything is great. Then they hire Tom and add him to the third-party team and everything is still great. Then they move Tom to the special finance team. Then Tom is fired and removed from the special finance team. Soon enough they log a support ticket that looks like this:

"We received a sales lead for Joe Customer from our third-party provider Bilk-Em Unlimited and it was assigned to our agent, Tom, who was fired a while ago and isn't on the third-party team. We've also disabled the Agent-For-Life feature. Please fix."

We actually receive a lot of these kinds of technical support inquiries. It's usually not the result of a "failure" in the lead assignment routine, but rather simple confusion about how leads are assigned and why they got assigned to a particular agent. This happens several times per week, if not daily. So if it's working "correctly", why do we have this deluge of technical support tickets?!

Our new lead on Joe Customer essentially makes him the invisible man. He's out of the loop. Agents don't see him, because he's not assigned to them, particularly when Tom doesn't work there anymore. And if agents don't see him, then they're not working the lead or earning their commissions. The "well oiled machine" our client wants is slipping a gear simply because they can't accurately predict when or why sales leads will be assigned to a particular agent.

This brings us to problem number two. Problem number two is actually described in some detail in a book I'm reading recently titled Influencer. This is a really good book. The problem in particular that I'm referring to is one of coping. As individuals and even as groups we tend to be much better at coping with an ongoing problem than we are at changing the status quo. It's in the words of the famous Serenity Prayer, "God, grant me the serenity to accept the things I cannot change, the courage to change the things I can and the wisdom to know the difference". The problem is that we all too frequently simply assume that we can't change a particular condition (in this case, constant support tickets for software that's working "correctly"), and so we focus too much on "the courage to accept" instead of focusing on "the wisdom to know the difference".

This is actually a much more common problem than you might think. Even amongst people like us who spend our 40hrs or more professionally changing things, we still frequently fall into the trap of thinking "I can't change that". And I want to address very specifically that I think more often than not, we fall into the trap of thinking "we can't change the user's behavior". To some extent I think as programmers we condition ourselves to think that the users are just tragically incompetent and can't really be helped. That's essentially been a mantra at more than one of the companies where I've worked. Hence nobody bothers trying to improve the status quo because everyone's assumed that any effort to do so would automatically fail. And then we fail by omission - by default - because we failed to try.

In our case for the moment, I'm grateful that I'm being given the opportunity to work on this particular problem. In this specific case, the real problem is not that we have tragically incompetent users. Although we may have a few users who are tragically incompetent, incompetence isn't responsible for a deluge of technical support tickets like this one. No the real problem is that we've turned Joe Customer into the invisible man. Yet every time I have a conversation about this with my current boss, what I hear from him is "before, when these tickets came in, I could usually tell what happened pretty quickly", in a manner of sort of oddly pleading for the "good old days" when we could handle massive loads of support tickets for working software. Say what?! You want these tickets?! He doesn't... but by now he's conditioned to wasting time on them instead of solving the problem.

Up to this point any time one of these tickets came in from a client asking about how or why a particular lead is assigned to a particular agent, the other guys on my team would just manually comb through the data to figure it out and explain it to them. It's unfortunate that it hadn't really sunk in that they were spending so much time on such a simple issue and that if they'd taken the time to address the fact that Joe was becoming invisible, they'd have eliminated a large number of those tickets that were taking away so much of their time. As with health or car trouble, often an ounce of prevention is worth a pound of cure.

Imagine for a moment that you've got a big wooden board and on this board are a series of clear plastic tubes into which letters can be placed. Now every single letter is vital to your business, and most of the time everything works great. Letters go into the tubes and the people who need them can see them immediately and grab them out of the tube and work on them. Every once in a while a letter somehow finds its way into a tube on the back side of the wooden board. The system is designed for letters to be in both sides, but nobody really looks at the back side of the board. And part of the problem is that the tubes themselves aren't labeled, so nobody knows who's putting them in the other side or why they're being put there, they just know that those letters aren't being processed. That's bad for business. So the solution we want is to bring the tubes around from the back side of the board and put them on the front and give them clear labels, so that our system is transparent and stops making those occasional letters invisible.

Right now when our automated system assigns a lead to an agent, the information about that assignment only goes into one place - the "lead" table in the database. But the lead table isn't a historical table. And this is another pandemic problem for software, that it doesn't provide adequate historical information. The lead is assigned, but there isn't any information being logged regarding the reason why the system chose to assign it to the agent. Without that log, everyone who looks at it, including those of us programming the system, can only guess at the reason why a particular agent received a particular lead. Any number of things could cause that guess to be wrong, from the agents on a team being changed to number of additional leads a particular agent can receive (which changes every time they close a lead or are assigned a new one).

So the project underway is to add a logging system that will store key information about the lead, the agent, where the agent is located and where the lead was generated, etc. and what precisely the state of the system was at the time which caused that agent to receive the lead. Once that logging system is in place we'll add that information to the existing display for the lead history in our application, allowing the client to see precisely how and why each lead is assigned. This will make the process completely transparent to our customers, allow them to more easily control exactly how their leads are assigned, and free up all of our time to work on real bugs and feature requests.

Transparency makes the invisible man... visible. :)

The one other thing that this new system will require is a method of informing the clients when this new data isn't available that the missing information isn't a "bug" in our system. Here's why. If the client hires a new agent or heck a receptionist and they become accustomed to seeing our wonderful new system and then later open an older lead and where they expect to see "assigned to agent Mike on Special-Finance Team via round-robin for customized lead provider Bilk-Em Unlimited" they instead see nothing, they'll think (and rightly so) that it's a bug! Information is missing! It should be there!

We obviously can't go back and retrofit all our existing leads because we just don't know what the system's reasons were when they were created. What we can do instead is to provide them with a message indicating that "this information isn't available". Hopefully we'll find a better way of describing it, but there needs to be some indication that it actually is working and simply doesn't have that information because the feature hadn't been implemented when the lead was created.