May 14, 2014
NFL football games are a rat’s nest of RF activity. Broadcasters, security, and many other departments all use radios. Without proper coordination, these thousands of devices would quickly interfere and descend into chaos. An elite corps of broadcast engineers labors in the shadows of every NFL game to ensure competing RF devices in all parts of the stadium successfully coexist. They call themselves Game Day Coordinators. This week we had the pleasure of speaking with Karl Voss, Broadcast Engineer and Lead Frequency Coordinator at the NFL. Karl coordinates many of the NFL’s special events, including the Super Bowl, which some have argued is the most densely packed radio event in the world.
Tell me a little bit about your history, and how you came to work with the NFL.
Well, I am the local SBE frequency coordinator for the state of Arizona, and I’ve been doing that for, gosh, 27 years. And as the broadcast auxiliary SBE coordinator here, we do all the events that happen in and around Phoenix, Tucson, and others. When Super Bowl 30 came into town the NFL came to me looking to find out if I was interested in coordinating the Super Bowl, and I just stepped in and did it.
What is a broadcast auxiliary SBE coordinator?
The Society of Broadcast Engineers designates certain people to coordinate the frequencies of broadcast auxiliary services at live events. Broadcast auxiliary services are lower tier radio devices, like wireless microphones, that support higher tier media services, like TV and radio stations.
So the coordination program was in place at the NFL before you came on?
No, it wasn’t. I understand they had no end of grief in Miami Super Bowl 29, it was a complete RF disaster, which is why the NFL decided to coordinate the Super Bowl and other games thereafter. I just happened to be in the right place at the right time.
What happened at Super Bowl 29?
I have no idea. All I know is that, well, see, the NFL stadium at Miami is probably a mile from the main transmitter site for all the TV stations. So Miami is an extremely different city to coordinate, and I’m guessing it was the transmitter site that caused the problems.
Miami Dolphins' Sun Life Stadium.
What do NFL coordinators actually do?
Right now, our duty is to cram 500 MHz of users into 25 MHz of spectrum. Everybody seems to think that RF is their god-given right. And essentially the job of the coordinator is to make sense of that—to try and give as many people tools that they need to do their job within reason. There’s a lot for rules we have to use. We have to follow the FCC rules. There are some guidelines that we have written within the NFL, too. Because there is just so much RF that shows up at a football game that you have to set up some sort of priority of use, beyond what the FCC has already defined.
How many devices are we talking about here?
Last Super Bowl, there were over 4000 entries in the database.
What type of database?
This is a proprietary database.
Is this an excel spreadsheet, a relational database?
It’s a SQL database. It’s a high end relational database. There’s a lot of folks that use a simple excel spreadsheet, and that’s OK. But what I do with my database is keep track of all the TV stations that are on the air, and of all the standard shipping frequencies that I’m aware of. So if you come up to me and say ‘I want a wireless microphone at Lectrosonics block 21,’ I can go right into the database and pull up whatever frequency your device is tuned to. And there are also a lot of end-users who have no clue how to operate their equipment or even what frequency they’re on. All they know is their switch setting is 7F. That’s all documented in this database.
What types of devices do you keep track of?
Wireless microphones, IEMs, camera control, telemetry, intercoms, digital cameras, walkie talkies. And at the Super Bowl we manage all RF, not just the broadcast auxiliary devices. We manage public safety, stuff for the stadiums, basically anything D/C to light that is allowed inside the NFL venues.
It’s not just a technical feat, it’s a social one too, right?
Oh yeah. I would say that coordinating is probably 75% politics, and 25% technical. You have to know the technical in order to successfully do it, of course, but I will go into the city ahead of time and do all sorts of seminars. What to expect. How the coordination process works. I always give the local broadcast folks a heads up. They get first shot at frequencies. And that’s because we’re on their home turf. I ask them to supply me with all the frequencies they need, and then essentially divvy up the stuff to give local folks first shot at it. That doesn’t mean the locals will get everything they’ve asked for. For example a TV station might have 100 wireless microphones, if they give me a list of 100 mics, they may get four. And that’s just to be fair to everybody. But it all starts with training. You basically train the folks on what to expect and how the process works, and then it’s essentially number crunching after that.
When you talk about 'folks', you’re talking about anyone who's going to be using any sort of RF. But there have to be some people who you just can’t reach, plus the unlicensed frequencies.
Well, that’s true, but for the most part we know who's gonna be on site because we credential them. You can’t get into an NFL venue without credentials. And there may be stuff that shows up that might be a surprise. But we’re really good at hunting that stuff down, too.
Walk me through what happens on a typical game day.
The first thing that happens is you pull together the database and assign frequencies to users. Then everybody who comes to the event has to go through a common entry point. At the Super Bowl, that’s the media gate. You come through that, everybody stops, and are told to turn their transmitter on, and we verify they are on the frequency to which they are assigned. If they’re not, they’re put on that frequency. And also verify that the transmitter’s clean. That it hasn’t been dropped and is putting out spurious emissions. From that point on, every radio has got a tag on it. And those tags can identify where they can be used and when they can be used.
We then deal with problems as they’re reported. We’ve got folks out there with spectrum analyzers and direction finding equipment. And we’re very good at finding folks operating where they shouldn't be.
Have there been any particularly memorable or outrageous or embarrassing stories of conflicts that have happened to your coordinators over the years?
Oh yeah. I really wish I’d kept a diary over the last 20 years of doing this, because I could write books. I got involved between two federal agencies who literally were unplugging each other’s radios until I pulled the people from the two agencies and myself into a conference room and closed the door, and we hammered out a deal.
We also had users come in with an ENG crew, and wipe out the dress rehearsal for the half time show, and we track them down and literally cut their credentials off and toss them out of the building. It’s take no prisoners. If caught interfering with somebody, and you’re not coordinated, you’re out the door. And it’s your job to explain to your boss why you couldn’t do anything because your credentials got yanked.
We’re talking about people who are professional users of wireless equipment, right? Not just some Joe off the street.
Right. But, professional user doesn’t necessarily mean they know how to operate their equipment. It’s taken a long time to train the majority of the users with procedures. But that still doesn’t stop somebody from hiring a freelance crew who may or may not know the process, or, may decide ‘oh, you know, I just need to do this one thing I’ll just find a frequency.’ It’s those sort of folks that we’re good at finding and escorting them to the door.
Do you have an RF bouncer with you?
Usually we don’t need to. But, programmed into our radios is NFL security. So, if we need to, we can pull out the big guns.
Typical aforementioned "big gun" RF bouncer.
How has coordination at NFL events changed over the 20 years you’ve been doing this?
It’s become a whole lot tighter. We’re trying put more and more users into the same amount of spectrum. Now we do time division. I may have one frequency used by user A pre-game and first quarter. And user B will have that same frequency half time, second quarter and third quarter, and user C has it fourth quarter and post game. There are zones where I may have the same frequency split between user A in zone 1, and user B in zone 2 and those zones are physically separated. There are time when you are restricted to certain parts of the stadium.
Also, a lot of the manufacturers are doing “frequency coordination” a lot in their software, that will constantly scan, looking for an open channel. But that is not really frequency coordination. Because all that does is look at the spectrum and look for areas where you can slap transmitters in. That is only a single user finding an area to work in. Coordination is about integrating all the users at an event, not just one.
How does one get to become an official coordinator for the NFL?
Initially, NFL coordination was tightly integrated with the SBE coordination program. So the majority of the NFL coordinators were SBE coordinators. Over time, that changed. If we lose a coordinator, we’ll look at other broadcast folks in the community around the stadium to find out if there’s somebody who might be interested in it.
Do you have to be a football fan?
Actually, it works best if you’re not. I’m not a football fan. You don’t get starry-eyed if you’re not a football fan. You just go in and do the job.
