Good tools won’t save an MSP without leadership

There’s a funny thing that happens when a real crisis hits an MSP: Everybody gets busy.

Phones start ringing, teams starts lighting up. Somebody is checking backups, another person is calling the client and someone else is digging through firewall logs. Somewhere in the room somebody is asking, “Has anyone checked the EDR?”

You can have a building full of smart people doing a whole lot of work and still have nobody in charge. Busy and organized are not the same thing; I learned that one the hard way.

I was standing on a stage at a TruPeer meeting getting recognized for our company’s growth when I found out our largest client was gone. Bankruptcy. Just like that, nearly half of our recurring revenue disappeared. We also had more than $200,000 worth of equipment inside a building we could no longer access.

I remember thinking, “Well, this is one heck of a way to celebrate growth.” It wasn’t ransomware or a tornado, but for our business it was absolutely a disaster.

The first thing I needed to do wasn’t fix anything: I needed to lead.

The worst time to pick a leader

Most MSPs have some kind of disaster recovery plan. We know where the backups are… Hopefully. We have documentation, escalation numbers, RMM, EDR, SIEM, BCDR and enough dashboards to make NASA jealous.

But ask a simpler question: Who makes the call?

Who decides this is no longer a normal ticket and has become an incident? Who talks to the customer? Who calls insurance? Who has authority to shut down systems?

If the answer is “Well, it depends,” you might not have an incident response plan; you might just have an expensive pile of tools. The middle of a crisis is a terrible time to start figuring out the org chart.

When everybody owns it

Small MSPs are prone to this because we’re used to everybody wearing ten hats. The owner can jump into a firewall, the service manager can take a support call, the senior engineer knows half the customers personally and the cybersecurity guy may also be the one who knows why the coffee machine makes that noise.

That flexibility is part of what makes us good, but it can also bite you when things go sideways. If five smart people are making five independent decisions, that doesn’t mean you have five times the leadership. You have five cowboys riding in five different directions. Somebody has to own the incident.

That person doesn’t have to be the smartest technical person in the room. Their job is to keep the big picture straight: What happened? What do we know? What happens next, and who owns it?

Let your best engineer engineer, let whoever is talking to the customer focus on the customer and don’t let the CEO turn himself into the most expensive Level 1 technician in the building just because something caught fire. I may or may not have learned that last one personally.

Urgency is not panic

One thing the military drilled into me was that urgency and panic are not the same thing.

Urgency says, “This matters. Move.”

Panic says, “DO SOMETHING!”

Sometimes the best thing you can do in the middle of an incident is stop everybody for sixty seconds. What do we know? What is affected? What could make this worse? Who owns the next move? Then go.

That minute feels painful when a customer is down, but I would rather lose sixty seconds getting everybody pointed in the same direction than lose six hours undoing a bad decision.

Your customer is watching

Customers don’t need to understand every technical detail.

They understand payroll can’t run, that nobody can get into email and the big scary ransomware note on the screen. More importantly, they understand whether the person on the phone sounds like they have a plan.

Technical people have a bad habit of waiting until we know everything before we communicate. During a real incident, that may never happen.

 So say what you do know: “We’re actively investigating. Here’s what we know right now. Here’s what we’re doing next. You’ll hear from us again at 2:30.”

Don’t promise an outcome you don’t control. Promise the next communication and keep your word.

Your people can break too

Your employees are part of the disaster plan.

A serious incident may run12-hours, 24-hours or longer. Your best engineer can survive for a while on caffeine, beef jerky and pure stubbornness—that doesn’t mean he should.

Eventually tired people make dumb mistakes. Even good, experienced people who normally know better. They skip documentation, misread a command or even get short with the customer.

So who takes over? Who looks at your hero engineer at two a.m. and says, “You’re done. Go home. We’ve got it”? A real disaster plan has to account for the fact that the people executing it are human.

Practice before it hurts

This does not need to be a fifty-page document. If it is, I’m betting nobody is reading it while the building is figuratively, or literally, on fire.

Pick your incident leader and technical lead. Decide who talks to the customer, who handles insurance and legal, as well as who can authorize shutdowns, restores and major changes. Then practice it. Tabletop ransomware. Pretend your largest customer is down, your office has no power, your PSA is unavailable or, here’s one that makes owners squirm, pretend you can’t reach the owner.

Good, that is the point. Find the holes while the disaster is fake; it is a whole lot cheaper that way.

Where I’ve landed

Disaster preparedness is not just backups, generators, cyber insurance and cybersecurity tools. All of those matter. But eventually somebody has to make a decision and has to lead.

Your team shouldn’t have to look around the room in the worst moment of the year and wonder who is in charge. They should already know who is leading, what their job is and what happens next.

My grandfather used to tell me a man is only as good as his word. I think incident response works the same way. Your plan is only as good as what your team can actually execute when everybody is tired, the customer is scared and nothing is going according to plan.

Anybody can look like a leader when the trail is flat and the weather is good. You find out who can really ride when the storm rolls in.

When the lights go out, somebody has to grab the lantern. Make sure your team already knows who that is.

Related: Why the most successful MSP leaders rarely grow alone