August 8–9, 2023 – A wildfire destroyed practically the entire community of Lahaina, Maui, incinerating thousands of structures, including the Old Lahaina Courthouse and the Waiola Church. Along with the tragic loss of life, financial damages could easily exceed $5 billion.
September 28, 2022 – With winds gusting over 100 mph, Hurricane Ian dropped more than a foot of rain from Florida’s Gulf Coast to its Space Coast, causing widespread flooding, power outages and over $50 billion in damages. Ian is considered the deadliest storm in almost a half-century.
May 22, 2011 – A devastating tornado slammed into Joplin, Missouri, reaching a maximum width of nearly one mile. The tornado destroyed over 4,000 buildings, including hospitals and basic infrastructure, and damage estimates totaled $2.8 billion – more than any single tornado throughout US history.
The recent wildfires and heartbreaking news from Lahaina, Hawaii, made clear that horrific things can devastate an entire community in literally the blink of an eye.
When fires, floods and tornados strike, there’s only so much you can control – but you can prepare for the worst by listening to those who have survived and acting on what they discovered.
We interviewed MSPs from across the nation to find out what they saw, how they responded and what they learned when their worlds came crashing down around them.
What They Saw
Mikol Westling, Maui Tech Solutions, Kahului, HI
Before the disaster, we’d received alerts about potential high-wind events due to Hurricane Dora passing 500 miles south of us. As an MSP, you think about wind-related items such as antennas, external access points and potential power outages, so we were prepared for those threats.
We weren’t prepared for a firestorm with 60 to 80 mph winds, caused by a hurricane so far away you’d have to scroll down to see it on the weather map.
That morning we were alerted to a road closure due to a fire, but within a couple of hours, they said the fire was contained and the road opened back up. Later that day, however, road closure notifications started again in various parts of the island. As more fires broke out, the magnitude of the situation became apparent, and we went into survival mode.
Between the size of the fire, the winds and everything else, you just feel helpless with all the uncertainty, because with the degraded communications of no cell service or Internet and everything else, you really have nothing. And as the hours of uncertainty turned into days of minimal information, misinformation, fake news and more, it became even more unsettling. You didn’t know if your home, your business, your staff or anyone you knew had made it out safely.
Anne-Marie Lerch, Hi Tech Hui, Honolulu, HI
This was a spot we’d visited every year for years. Our good friend has an art gallery we visit in that area. We’re very familiar with Lahaina, and we have clients with businesses there.
Initially, when we heard there was a fire, we didn’t know how bad it was. But as the news progressed, we began realizing how awful it was. We started texting back and forth with friends, and that’s when they told us how dire it was.
We had a friend riding on a Harley in town, and he only escaped by driving up the sidewalk. He said his back felt like it was literally on fire.
One of our employees had cousins visiting from New York, and they were on the street when the fire broke out. While they were driving their rental car, people were knocking on the window asking for rides. They eventually had to abandon the rental car before it caught on fire and then walk through the ocean to get past the fire to get to their hotel.
It’s all burned down. So we weren’t directly affected, but many of our employees and our clients were. We never would have thought it would happen so close to home.
Matthew Rebstock, Tech In A Flash, Cape Coral, FL
Honestly, when it’s hurricane season, you get them and that’s just how it is. It’s something that’s common so you prepare for it – put up your shutters and make sure you’ve got enough water and supplies. And then that’s it.
But in the case of Ian, It was all kinds of nuts. It was bad. The first hour into it, the power goes out and you’re stuck, listening to some of the rawest, most intense noises you could ever imagine. I poked my head outside and watched plastic sheds and whatnot tumbling down the street. It just ripped everything apart. Electric light poles twisted and snapped off like twigs. They had to rebuild the entire electrical grid.
When hurricanes come, there’s a storm surge and flooding that happens. Luckily, in the area I’m in, we didn’t have a storm surge. But we had clients who had eight feet of water in their homes. In Fort Myers Beach, the water went up to the second story. Imagine being in the middle of that and having to deal with that.
The hurricane left nearly 98% of Lee County without electricity. We had running water, but it wasn’t sanitary. Internet was out, and because the cell towers had been snapped like twigs, there wasn’t any cell service either.
John Motazedi, SNC Squared, Joplin, MO
The tornado happened around 5:30 p.m. on a Sunday, and literally from the time the tornado sirens went off, we had about 15 or 17 minutes’ notice. This one just felt different than the others. The sky was darker and greener. It was just weird, and ultimately it was all over in minutes.
We’d lost power, and I told my wife, Renee, “We’re going to run to the office really quickly because I’ve got those magnetic locks, so we need to lock the doors.”
The office was maybe half a mile south of the tornado’s path. We didn’t really know what was going on. As soon as we turned the corner, it looked like a war zone. I could see the hospital, and all the curtains were streaming out the windows from where the wind was going through. Cars, trees, telephone poles on the street – it was a mess, to say the least. We helped take some injured people to the hospital.
Ultimately, I got to the office later in the day; it was getting dark. There were no landmarks you could recognize. All the trees were down, the telephone poles were down, the signs were down, the buildings were gone. All the street markings were gone.
I’m walking, trying to figure out where I am, thinking, “My office should be around here somewhere.”
And as I looked more carefully, I saw the outline of a parking lot and I realized I’d been standing in my office’s parking lot. I hadn’t recognized it simply because there was nothing there to recognize.
Everything was gone.
How They Responded
Mikol Westling, Maui Tech Solutions, Kahului, HI
We were extremely fortunate in that we didn’t lose our business or our homes.
Our first order of business was to identify those who were affected, triple-check that all client backups had been completed and start implementing the BCDR plans we had in place for our customers.
When we began getting calls from customers, friends and family who’d been evacuated and were looking for a place to camp, we opened our doors to them. We set up a small command station with additional power outlets, charging stations, cables and an additional access point so people can recharge, connect and let loved ones know they’re safe.
We’ve also been focusing on our customers and on helping others. We reached out to our JV partners, such as Cytracom, Dell, HP and T-Mobile, and arranged for special rapid deployment, provisioning and replenishment programs. We also let our competitors know that we’d offer our services to them at cost for tools, VoIP, etc., and let them mark up as they see fit so they have the opportunity to generate some revenue. We didn’t even charge for labor.
This allowed our competitors to continue to service their customers, so they could then focus on their lost homes, businesses and other stuff that comes with the aftermath.
I’ve been doing fundraisers and helping check out different charities that friends, family and customers have asked me about. They know that, being a geek, I can usually vet out whether it’s a valid charity or not – and that matters because there are now over 50-some phony charities.
Anne-Marie Lerch, Hi Tech Hui, Honolulu, HI
Because it was a different island, with access so limited, there wasn’t a lot of hands-on that we could do.
Right away, we called our clients to make sure everybody was okay and asked what they needed.
One of our clients is a nonprofit that is directly helping Maui, so we promoted donating to that nonprofit. However, the problem was that the funds weren’t getting to the people who needed it fast enough and the organization wasn’t able to help get help to them.
The area’s still blocked off, they won’t let anybody in that area, so it’s hard to get supplies to their homes. But average people are just getting together and doing what’s needed. We have friends taking their own boats from Honolulu all the way to Maui with supplies that are needed. That way it’s going into the right hands and to what they need right now.
Initially, everyone was donating clothes and stuff. But they actually said, “Stop doing that,” because it’s a logistical nightmare. They didn’t need that, because their houses had burned down, and they had nowhere to put anything. Right now they need masks, because the fire produced so many chemicals in the air that they can’t breathe. And we just delivered those.
So, it’s a matter of trying to figure out exactly what they need, and then giving it to the people who can actually deliver it.
Matthew Rebstock, Tech In A Flash, Cape Coral, FL
We were very lucky. While some went a month without electricity, we had power and Internet within the first week. We opened up to the general public, not just to our clients, and set up charging stations for cell phones. It was almost like an Internet café, where people could come and connect to the rest of the world – if they needed to do business or whatever they could do out of our office.
One of my clients had about eight feet of water in their place. When I went to check on them a couple of days later, they’d already gutted everything down to the studs. She said, “I’m so thankful that the computers didn’t get destroyed.” I said, “I know you’ve got to go through this whole rebuild, so let’s take them back to my office and just make sure they work so that you’re not dealing with it when you’re trying to get back up and running.”
We get them back to our office. That night there was a gigantic rainstorm; it caved her roof in, destroying absolutely everything. But we had saved her computers and we were able to hook them up in our place so she could run payroll and do all that kind of stuff.
John Motazedi, SNC Squared, Joplin, MO
We did some text messaging to make sure all my employees were fine. Our initial goal was just to help people; it was no longer about computers.
The following morning, we got together with the goal of getting back up and running. Obviously, with the office gone, everything’s gone – including all the servers and the racks and so on. But we did have a disaster plan. We got our servers back up and running in the cloud, so we could tell which of our clients were functioning and which were not. We realized we had lost about half of our client base in that tornado, about 400 workstations out of the 1,000 we were managing.
We started calling our clients on cell phones and telling them, “Hey, I know you’re down, but we can get you up and running.”
A good portion of our client base was physicians, so we wanted to make sure we were meeting all the HIPAA requirements and compliances, and protecting patient information on those machines that might be damaged or et cetera.
We also wanted to make sure their data was protected and they had all the backups with no issues. We got them back up in the cloud and functional. (I think the last client backup got to the cloud around seven or eight minutes before the actual tornado.)
We wanted to make sure patient records and information were safe and not on servers that had been crushed or damaged. We found one server that was continuing to run because it was on a battery backup even though there was no building around it. When we took the lid off the server, it had been sucking in all its surrounding insulation, so it was a mess – all mixed up with wires and cables and all that stuff.
A lot of people came in and helped us. Robin Robins got the message out to the TMT community, and I think we got well over 30 or 40 different people at different times helping us. It was definitely a team effort, but for that first six weeks, it was literally long, long days and lots of hours.
Lessons Learned
Mikol Westling, Maui Tech Solutions, Kahului, HI
Prepare, prepare, prepare!! Just like you would prepare for a hurricane, tornado or some other natural disaster for your home. Make sure you have clothing, water, a generator, fuel, etc.
Have a go-bag/box/tote of essential IT equipment that you or your customers might need. A few lightweight laptops, thumb drives or external drives, easy-to-set-up wireless access points, etc.
Make sure you have up-to-date copies of BCDR plans and contacts both in printed and digital form so you can access them quickly when needed.
For our TMT community, keep a close eye on the Que – it’s an incredible resource when you’re looking for help, inventory, etc.
And if you’re an MSP that’s near but outside of an affected zone, reach out to your competitors and pick up the load for them. Yes, you’re competitors, but who better to help than a fellow MSP? Yes, it WILL cost you money if you do it the way I did, by offering it up at cost, but wouldn’t you want someone to do that for you if you were in this situation?
We’ve seen acts of heroism just in assisting people. My best friend swam in fuel and water and other stuff on air mattresses. As soon as he hit the beach and unloaded it, he gave the air mattresses to some people who were displaced and just camping on the beach.
There are all these acts of kindness happening right now. It’s pulled us back together as a whole community, as a whole island.
Anne-Marie Lerch, Hi Tech Hui, Honolulu, HI
God forbid, you don’t ever think you’re ever going to need it, but you’ve got to have a disaster recovery plan in place in case something like this happens. Because you never know. There’s no warning when a disaster is going to hit.
We had one client who had lost their whole store, but they’re still in business. It’s not stopping them, thanks to cloud first and zero trust. You know, all the famous buzzwords, but it does make a difference. We were already cloud first before the pandemic hit – so when we had to work from home, we didn’t miss a beat.
Right now, we’re going to our policies and procedures and testing them. What are we going to do in a disaster? We’re testing and documenting and training for all that.
This is a good wake-up call. And even if you already have a plan, go back through it right now. Revisit it and start testing that plan.
For instance, what’s going to happen when electricity is out? What’s the plan?
Where are people going to go work?
How are you going to still function?
We’re an MSP, and we’re a cyber security company, so we can never be down. That means we’ve got to have generators in place or there’s going to be Internet and a reliable data center – we have to make sure we continue to function.
So, make sure you’re testing the plan, documenting it, and it’s being trained across the team.
Matthew Rebstock, Tech In A Flash, Cape Coral, FL
When discussing disaster planning, every MSP always talks about fires, hurricanes, earthquakes, whatever. But face it, the majority of the problems their disaster recovery plan and process is going to have to deal with involve the ins and outs of doing business every day, like the e-mail getting compromised by an employee, someone deleting the wrong link or whatever. It’s all about education, and if you start with the natural-disaster stuff, you’ve got to lead from that to the reality that the user is the biggest risk you constantly face.
We did tabletop exercises with our clients where we went through a disaster scenario, or we pretended there was a ransomware event, or whatever. It’s almost like you’re playing Dungeons and Dragons, right? When you do that, you realize that you have to have the key players in the room with you, going through the process step-by-step.
By the time you get to the end of that scenario, you can see where this or that in the plan is incorrect and then go ahead and update it. After an incident, you definitely want to go through a post-mortem, but make sure you go through the plan and practice it.
Finally, the Technology Marketing Toolkit community contacted me all throughout this thing. People offered to provide help-desk support, phone triage and even labor to run the business. People in the community actually started a GoFundMe page for us. They donated money, and that allowed us to make sure that all of our employees were being taken care of on a personal level before they actually had to come back to work.
This community is amazing, and the outpouring of support I had from all of them was amazing. It restored my faith in humanity. You get to see the best of everybody, with everyone pitching in to help.
John Motazedi, SNC Squared, Joplin, MO
This would not have been successful without all the people who stepped in and helped – all the volunteers, Robin Robins, the TMT community, all those people who helped us in their own way – it was incredible. So don’t hesitate to reach out for help.
Have a plan in place. Test your backups. Nine months before the tornado, we had an issue with a backup that didn’t automatically tell us the backup had failed. So I immediately assigned one of my engineers to spend time every day checking every backup on every server. The first four hours of every day was dedicated to checking those things; we started checking those things on a regular basis.
Ultimately, at the end of the day, you’ve got to have good people working for you. And we’ve got a great staff and we had a lot of great people who came to help. To this day, those eight to 12 weeks after the tornado were probably where we were at our best – anything you can imagine, from logistics to scheduling, planning… I don’t know how to really describe it. Everyone was laser-focused, everybody knew what they had to do.
It’s funny, and I don’t want to ever have to go through a tornado again, but that level of teamwork, logistics, coordination, how well we were performing, how happy the customers were, how happy our staff was, et cetera, it was amazing. I’ve never seen anything like it and will probably never see anything like that again.
