By Kelly Hill

The ability to automatically, virtually and in parallel test new 5G Standalone services, slices and software updates in the cloud is key to Dish Network’s network strategy and its differentiation, according to Dish EVP and chief network officer for its wireless business, Marc Rouanne. Rouanne said that the ability to rapidly test and certify network software and services has been part of Dish’s vision for its network. With the announcement yesterday of Dish’s selection of Spirent Communications to enable automated, large-scale 5G core testing, Rouanne offered up some new insights on how Dish plans to leverage the swift and virtualized cloud-based 5G testing capabilities that Spirent is providing.

Dish says it will be the first telecom company to run its service on the public cloud, and it is partnering with AWS on cloud infrastructure. It needs to be able to test and validate its 5G network functions and services on AWS infrastructure — and in July of this year, Spirent had announced the capability to do just that, with a new Landslide 5GC Automation Package that the company said at the time is “designed to help carriers to rapidly deploy 5G networks on AWS.” The package combines AWS’ Continuous Integration and Continuous Delivery (CI/CD) pipeline with Spirent’s 5G test capabilities to enable carriers to “objectively, rapidly, and continuously” validate network functions and services, according to the test company. Doug Roberts, SVP and GM at Spirent, said that the 5G automated core testing is made up of Landslide, plus Spirent’s Velocity and iTest automation framework to produce a “single, automated test library specifically for 5G Standalone and NonStandalone applications.

The capabilities that this supports, Rouanne says, are “extremely disruptive — more than what anybody thinks.”

Dish, he says, made a strategic decision early on that its testing and lab strategy would be virtual. “The telco industry is a very slow industry, because of the way that labs are set up between the vendors, the operators and then the go-to-market, where there is a very long lifecycle for moving software and capabilities from the vendor development all the way to in-service,” he explains. That timeline is average around 12 to 18 months, Rouanne says, although he acknowledges that some “pockets” may move faster. This, he continues, has “made the telco industry vulnerable to attacks from the over-the-top [players],” and so major applications like messaging or video conferencing “have been hard for the telco to keep in the domain of the telco”. That software development velocity has advantaged the OTT ecosystem.

“We decided that with 5G SA which is a service-oriented architecture, that we could do something different,” Rouanne says. “A very important input in our decision to go to the cloud with AWS was that we wanted to not only have the network in the cloud, but all the testing capability.

“We can spin off a new testing environment within ours, which is kind of building a new lab,” he continues. “Imagine I want to test security end-to-end. I can spin off a new lab, end-to-end, that is on the scale of a sub-network, that is much bigger than any lab that an operator would have — I can do that within hours. I can do another one for roaming testing. I can do another one for some vertical testing. And it’s all automated.”

He goes on to explain that Dish has asked all of its vendors and partners to be come and test in those virtual labs. That is important, Rouanne says, because the various labs used in the stages of telecom software development aren’t necessarily representative of the network where the software will ultimately be used. They have different attributes, platforms, paths or perhaps use another Kubernetes layer. “They look the same, but they are different,” Rouanne adds. “Which means that when you move the software from a vendor lab to an operator lab, you have to re-engineer everything. And usually it takes weeks just to do that, before you can start testing — whereas natively now, with Dish, we ask our vendors to test in the lab in the cloud, in the replica that is exactly the same as our network. We have the same stacks, we have the same networking, sub-nets, security, everything is the same. So when our vendors are testing, they are testing in a virtual lab that is just part of our network. That’s a big thing.”

Similarly, he says, Dish’s virtual testing strategy also plays into its strategy to be able to offer one of 5G Standalone’s most highly anticipated capabilities for enterprise: Network slicing.

“Our network is designed with slices in mind, which is very unlike any other network today,” Rouanne says. “The problems of slices is that if you’re not thinking life-cycle management for slices, you can’t just spin them off. Our ambition is to be able to offer one or many slices to any enterprise. Each enterprise could have many slices: Some high-speed slices, some very secure slices, some monitoring slices. … The problem is that they will have different requirements, so they will have different features. They will have different capabilities. Some may optimize for latency, some may optimize for cost, some may optimize for reliability or redundancy or security — or all of it. Which means that the testing of the slice has to be specific.”

Such testing in a traditional lab would be sequential and highly manual, Rouanne says: Test one slice, probably for several months; test a second slice for several months; and so on, changing configurations within the lab between rounds of testing. “Because we test in the cloud and because we can spin off slices or testing environments live, we can maintain testing environments in parallel for different slices and different verticals,” he says. “If you don’t have that capability, I would guess that you are going to be able to do a few, a handful of slices — two, three, four. But you won’t be able to slice per enterprise, so you won’t be able to deliver what they really want: Their own sub-network that they master.”

But running slices and parallel test environments at that scale requires massive automation, Rouanne goes on. Most operators, he says, are running a large part of their testing manually, because they’re doing it sequentially with a limited number of parameters. “At the scale we’re talking, with all these environments in parallel and slices, there’s just no way that you can do it manually,” he said. From day one, he continues, Dish has told its vendors and software providers that it will require everything to be tested in the cloud, and ultimately, be able to create tickets automatically and virtually, log correlations in network behavior, and so on, in active network operations. “It’s like automating a factory,” he adds, saying that Spirent ultimately had “the right mindset and ambition” to be selected as Dish’s partner in doing so.

For Spirent’s part, Roberts explains that the company has been on a multi-year journey of transforming its traditional box-based business to the cloud. “What we realized, as a company … is, the cloud is here,” Roberts said. In parallel, he says, “5G in and of itself fundamentally changes the very infrastructure and the architecture itself.” It’s no longer an option to pick a custom-built chassis and plan to update it a few times a year. That means testing infrastructure has had to change, right along with the networks themselves. “If you think about the testing infrastructure, whether in the lab, whether pre-production or wether live in production, it really goes back to the monetization of the model of the lab itself,” Roberts explains. “We used to spend 90% of our time just simply building the lab out: What’s the infrastructure, how much rack space do we need, the HVAC componentry. This is regardless of whether it was a co-located data center or a self-owned, self-managed data center hosting the lab.” By having replicable virtual labs hosted on AWS infrastructure, Dish aims to avoid most of those physical-hardware-related questions and focus on testing the software and services in a far faster timeframe.

5G also means working with vendors who have different cadences for updates. Some have new releases every week, Roberts points out. “We’re now facing testing and certification demands that in some cases actually run and roll on two to three day cycles,” he said. “So the ability to spin up, on-demand, a lab infrastructure with fully-vetted 5G test emulation and certification capabilities, in a matter of minutes, is literally the new standard, the new table stakes … in the industry. However, I’m very cognizant that just a year ago, it sounded like science fiction.”

Rouanne says that Dish had to start with the much more manual processes that existed at the time, and is now programming all of them into Spirent’s system as the company moves toward deployment and a new stage for its network.

“I think this is the tip of the iceberg of something which is much bigger, which is all this automation and access to data,” Rouanne says. The partnership with Spirent, he says, “is taking us to the next level in terms of agility and speed. … We have always planned for automated testing and capabilities, it’s key in our deployment. We have always said that we want software life-cycle and innovation life-cycle to be two big differentiators for Dish. So Spirent is key in bringing that to us.”