Aug 2, 2022

The number of ransomware attacks fell 20% sequentially in the second quarter. Photo: Eclipse Images via Getty Images August 2, 2022 Benjamin Pimentel ( @benpimentel ) covers crypto and fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Google Voice at (925) 307-9342. August 1, 2022 The crypto crash, which has wiped out about $2 trillion in value and upended a once-fast-growing industry, has an unexpected twist: Even criminals are feeling the pinch. Ransomware attacks have dropped sharply this year as perpetrators grapple with an economic downturn, the Ukraine war and the dramatic plunge in the prices of cryptocurrencies they’ve been routinely using to commit crimes. “There is never any single reason why anything happens in cybersecurity [but] in this case, the thought that volatility in the crypto markets is a contributor to the drop in ransomware attacks makes sense,” said James Lee, chief operating officer of the Identity Theft Resource Center. The number of ransomware attacks fell 20% sequentially in the second quarter, the first quarter-on-quarter drop since the ITRC began tracking ransomware attacks in 2018, the nonprofit organization said. It’s not just the drop in the value of crypto. Enforcement efforts are having some impact, according to SonicWall , which recorded 236 million ransomware attempts globally in the first half of 2022. That’s down 23% year-over-year. On top of falling prices, “increased government and law-enforcement focus impacted both who cybercriminals chose to attack and how well they were capable of carrying out those attacks,” SonicWall said. Ransomware had become such a big problem that the Biden administration last year urged U.S. businesses to focus more on securing their networks. Crypto’s rise has made curbing the attacks more challenging. Lee said that, since 2018, cryptocurrencies have been “the preferred method of monetizing ransomware attacks because of the difficulty in clawing back funds and the — up to recently — ever-increasing value of the coins.” Last year, the Justice Department recovered $2.3 million in bitcoin ransom paid to DarkSide, the criminal group that hacked Colonial Pipeline. The DOJ subsequently announced the creation of a crypto enforcement team to go after criminal actors using cryptocurrencies. Besides the drop in attacks, there are other signs of declining interest in laundering ransomware proceeds through crypto networks. Kenneth Goodwin, director of regulatory and institutional affairs at Blockchain Intelligence Group, said the crypto compliance and forensics company has recorded a decline in mixers typically used to obfuscate blockchain transactions, especially in illicit transactions. It’s important to note that accumulating cryptocurrency itself is “not the end goal” of ransomware perpetrators, said Mark Manglicmot, senior vice president of security services at Arctic Wolf. After the victim pays the ransom, the criminals typically seek to convert it to fiat. That becomes trickier “with fewer outlets for disposing of cryptocurrencies due to bankruptcies and reduction in crypto value,” Lee said. “It makes sense cybercriminals would look for other ways to make money that involve less risk.” Price volatility clearly poses a problem for ransomware criminals, said Alma Angotti, a partner at Guidehouse. “They could just ask for more bitcoin, right?” she told Protocol. “They could just ask for 20 bitcoin instead of 10 or whatever. But if the price is gonna drop even further after they get it, that's probably a factor.” The crypto slump is definitely not the only factor, Angotti said. Many companies have also balked at paying up in ransomware attacks “because their insurance companies may not cover it,” she said, or they could get charged for violating the law. “You could now also be hit with a sanctions violation besides having the money that you lost to the ransomware, so that's a problem,” she said. Manglicmot argues that the Ukraine war probably plays a key role in the decline. “A lot of the threat actors are known to be based in Eastern Europe,” he said, which leads him to suspect that the decline in ransomware attacks is “likely because of where the attackers are based.” Not everyone is convinced the data shows a long-term trend. Sam Curry, chief security officer at Cybereason, said the more recent dip in ransomware attacks “might also have to do with summer slowdowns in IT — and people who might otherwise click on the wrong thing might just be on the beach with their families.” Rick Holland, chief information security officer at Digital Shadows, agreed, saying “any perceived slowdown in extortion” should be considered “as a blip, not a trend.” “The summer months typically see slower extortion activity,” he told Protocol. “Criminals take vacations too.” Keep ReadingShow less July 27, 2022 David Silverberg is a Toronto-based freelance journalist, editor and writing coach. He writes for The Washington Post, BBC News, Business Insider, The Toronto Star, New Scientist, Fodor's, and several alumni magazines. He also writes for brands such as 23andme, Shopify and Bold Commerce. He has served as editor of B2B News Network, Canada's only B2B news magazine, and Digital Journal, a leading pioneer in citizen journalism. Find more about him at www.davidsilverberg.ca July 11, 2022 GREG CROSS (CEO, Soul Machines) GREG CROSS (CEO, Soul Machines) is one of the original tech nomads, spending his career traveling to and living in every major tech market in the world. He now lives in New Zealand but creates businesses that compete on the international stage. Most recently, PowerbyProxi, a wireless charging company he co-founded, was sold to Apple in 2017. In 2016, Greg co-founded Soul Machines to build a Human OS™ for Artificial Intelligence and explore the future of human-machine cooperation. Soul Machines is at the cutting edge of AGI research with its unique Digital Brain, based on the latest neuroscience and developmental psychology research. Partnering with innovative people and brands like Carmelo Anthony, Procter & Gamble, NESTLÉ® TOLL HOUSE®, Maryville University, and The World Health Organization, Soul Machines is re-imagining what is possible in the delivery and underlying economics of empathetic customer experience. Greg holds multiple chair positions, is the Sir John Logan Campbell Executive in Residence at the University of Auckland Business School, and was inducted into the New Zealand Hi-Tech Hall of Fame in 2019. Nicklaus meets and chats online with his Digital Twin in May 2022. Soul Machines co-founder and CEO Greg Cross and his co-founder Mark Sagar, Ph.D., FRSNZ are leading their Auckland and San Francisco-based teams to create AI-enabled Digital People™ to populate the internet, at first, and soon the metaverse. As this field has grown over the past six years, enterprise brands and celebrities have increasingly turned to Soul Machines to digitize their workforces and level up in how they engage with customers and fans. They humanize AI to create Digital People that take input from the environment — a question, a facial expression like a smile — and respond in real time. Digital People, such as the one used by Nestle to serve as a digital cookie coach on its website, allow brands to offer an empathic and ultra-personalized customer experience. Using similar autonomous automation technology, Digital Twins take the customer and fan experience to another level. The celebrity-based avatars boast lifelike features because a real person is captured, creating a “Digital Twin” of the star’s likeness. It can answer customers’ questions with responses that are aligned with the celebrity’s expertise, background and legacy. The entertainment and sports industries could benefit from developing interactive digital avatars, but the cross-pollination of virtual animation and AI must veer far from 2Pac-hologram territory. Soul Machines’ approach is layered with next-gen AI applications, such as its Digital Brain technology, which allows for natural-language processing and empathetic, responsive behavior. In layman’s terms, that means we could talk to these Digital Twins in real time, but in the entertainment world, that relationship could get even more compelling. Protocol spoke to Cross to learn more about their newest release, a Digital Twin of Jack Nicklaus, the retired golfing champ who’s won a stunning 117 tournaments. Depicting Nicklaus at 38 years old, his Digital Twin represents the potential of this technology, allowing fans to ask questions and hear stories from his 60-plus years on the links. Digital Twins will soon partner with retail brands (among others) to offer expertise and recommendations on products and services, as well. Cross takes us on a tour into a technology that may be nascent now but could soon become the competitive edge that sets successful brands apart from the rest. Extensive capture technology maps Nicklaus’ facial expressions. What motivated you to launch Soul Machines with Mark Sagar, and what makes your Digital People appealing to brands? I’m a serial tech entrepreneur, and I just came out of a previous business that sold to Apple. I started looking around for my next move and, through a mutual friend, was reintroduced to Mark. I had met him before, and he blew me away with who he is as a person and his commitment to his life’s work. He’s won two Academy Awards for the animation technology he built that was used in films such as “Avatar” and “King Kong.” We had a beer, and he talked about coming up with a new paradigm for animating digital characters, and Soul Machines began soon after, in July 2016. As for our Digital People, we see them as the future of customer and fan engagement. We’re living in an increasingly digital world, and the major challenge for brands is creating those personal connections with fans in a more digital world. And that’s where Digital People become important. We, as humans, are hardwired to emotionally engage face-to-face. Soul Machines technology can autonomously automate back-and-forth conversations that are each unique. We see Digital People being such an incredible way to create scalable customer interactions in digital worlds. What competitive advantage would these avatars offer to enterprise brands? If I create a digital workforce, all of a sudden, I’ve created a highly scalable workforce that is always on. Those customer-centric Digital People can have 1,000 or 100,000 conversations, and these are uniquely personal interactions that are hard to achieve and staff in the real world today. Conversational AI becomes that repository for the brand experience. Also, brands get a smoother consistency of experiences with Digital People who can retain all that data from those interactions. Especially as we move into metaverse worlds of tomorrow, adopting this technology will truly offer competitive advantages to brands. The Digital Jack Nicklaus avatar is fascinating to us. How did you create it? How did he react to the idea? We’ve always wanted to be in the digital celebrity experience space. We first worked with rapper will.i.am in 2019 by creating his Digital Twin for an AI documentary series. We wanted to test this concept further by having amazing CGI lead to hyper-realistic people who are autonomously animated to create the ultimate fan experience. We are in talks with a range of different celebrities. We enjoyed the enthusiasm Jack Nicklaus and the Nicklaus Companies have for moving the brand into the future. With Soul Machines, he wanted to extend his brand to the next generation of golfers. He wanted to be 38 again, when he was at the prime of his career, so we scanned him and his son Gary, who looks so much like him. The kind of storytelling engagement that will come from Digital Jack will build off all the tournaments he’s won and the many golf courses he’s designed. Share your vision for how you think the metaverse will mature in the coming years, and how Soul Machines will play a role in that maturation. We are only at the beginning of the metaverse. The hardware that brings it to life hasn’t matured yet, and isn’t defined as a tech stack now. The most important thing for us is to encourage brands to think about how investing in something today creates seamless experiences tomorrow. AI gets stronger and better with each interaction, and that’s why Digital People provide the most personable and scalable customer experiences that will live on the metaverse and elsewhere. We envision a digital workforce that can move seamlessly between 2D and immersive worlds, and that’s really exciting to us. Keep ReadingShow less Benjamin Pimentel ( @benpimentel ) covers crypto and fintech from San Francisco. He has reported on many of the biggest tech stories over the past 20 years for the San Francisco Chronicle, Dow Jones MarketWatch and Business Insider, from the dot-com crash, the rise of cloud computing, social networking and AI to the impact of the Great Recession and the COVID crisis on Silicon Valley and beyond. He can be reached at bpimentel@protocol.com or via Google Voice at (925) 307-9342. August 1, 2022 In charging a former Coinbase employee with insider trading, the SEC fired what appeared to be the opening salvo in a bigger legal brawl with the crypto powerhouse over a crucial question: Are crypto tokens securities? In fact, the fight was already brewing. Before the insider case arose, the SEC was reportedly already probing Coinbase itself for alleged securities laws violations, setting the stage for a confrontation that could define not just the company’s future but that of the entire crypto industry. Coinbase quickly signaled plans to fight back. The day the SEC announced its charges, which leaned heavily on the notion that the tokens involved were securities, Chief Legal Officer Paul Grewal flatly declared that “Coinbase does not list securities. End of story.” He also accused the SEC of having “little interest in this most fundamental role of regulators.” Coinbase executives have been known for unleashing strong, even fiery public statements against the SEC. But Grewal and Chief Policy Officer Faryar Shirzad, Coinbase’s generals in the battle with the SEC, also highlighted key arguments for the company’s legal counteroffensive. Of the nine digital assets the SEC insider trading complaint said were securities, seven are traded on Coinbase. Grewal said the company has “a rigorous process to analyze and review each digital asset before making it available on our exchange — a process that the SEC itself has reviewed.” That was a reference to the SEC review of Coinbase’s IPO filing last year, according to a company representative. The SEC declined to comment. It’s far from clear that the issues the SEC would consider in vetting a company’s financial reporting ahead of an initial public offering have much to do with the question of whether digital assets are securities, even if the same agency happens to have oversight in both cases. John Reed Stark, former chief of the SEC’s Office of Internet Enforcement, said Grewal’s statement was misleading. “What they’re saying implies some sort of approval,” he told Protocol. Marc Fagel, the SEC’s former San Francisco regional director, agreed that the company statement is “obviously a little self-serving.” “Whether the SEC did or didn’t review Coinbase’s listing process has no bearing on whether a particular coin offering required registration under the federal securities laws,” he said. As SEC chair, Gary Gensler has made his view clear that most cryptocurrencies are securities. But that stance predates his arrival — under Jay Clayton, the agency sued Ripple on the premise that XRP was an unregistered security. The long-standing judicial standard used by the agency, that it’s now applying to crypto, is called the Howey Test, which looks at whether people are investing money in something with an expectation of profits from others’ efforts. “What the SEC will say is, ‘These are the rules. They've always been the rules. It's technology agnostic, and we will use the test to decide something as a security,’” Alma Angotti, a partner at Guidehouse and a former SEC senior counsel, told Protocol. Coinbase has argued that current rules are outdated and don’t make sense. “Securities law is … not well-suited to govern digital assets,” wrote Shirzad, the company’s chief policy officer, in a proposal for a separate federal regulator in charge of crypto. That proposal didn’t gain much traction; the leading bill for crypto regulation from Sens. Cynthia Lummis and Kirsten Gillibrand would divvy up crypto regulation among existing agencies, including the SEC. The day the SEC announced the insider trading complaint, Shirzad announced that Coinbase had filed a petition “asking the SEC to begin rule-making on digital asset securities.” He said Coinbase is asking the regulator to “start a process where the public and key stakeholders can transparently provide input into the agency’s work on crypto.” The SEC has a rule-making process to make or update rules for laws passed by Congress that is “designed to give members of the public an opportunity to provide their opinions.” But it’s unusual for a single company to request that the SEC begin a rule-making process, experts say. Trade associations or Congress itself are known to make such requests. “I've never heard of anybody doing that before,” Angotti said. Stark said there’s “absolutely nothing wrong with asking the SEC to come out with guidelines, but the reality is in the area of crypto [it] brought over 100 cases” and has come out with “every conceivable form of guidance already.” The SEC has issued some rules that led Coinbase to make changes in its business. For example, the SEC announced early this year that companies holding cryptocurrencies on behalf of customers must record those assets as a liability on their balance sheets and disclose potential risks to investors. That prompted Coinbase to disclose in a regulatory filing that customers could lose their crypto assets if the company went bankrupt. And Coinbase has argued that it has gone to great lengths to comply with the existing rules. Testifying before a House subcommittee on intelligence and counterterrorism in January, John Kothanek , Coinbase’s vice president for global intelligence, noted that the company is a licensed money transmitter in more than three dozen states and is a federally registered money services business with FinCEN. Grewal said the company “cooperated” with the SEC’s investigation of the former Coinbase employee. But he complained that “instead of having a dialogue with us, the SEC jumped directly to litigation.” Omid Malekan, who teaches blockchain and cryptocurrencies at Columbia Business School, said there’s been a debate in crypto on “whether Coinbase’s mistake was trying to play with the rules because at every turn, it seems to result in more crackdowns, more blowback and you don't see a regulatory agency like the SEC praising them for at least trying, for going to them, for communicating with them.” If there’s anything to take away from Coinbase’s freshly combative stance, it’s that the company may think it has little to lose from confrontation. Keep ReadingShow less Th long-sought idea of shorting startups may be coming to fruition. Illustration: iStock/Getty Images Plus; Protocol August 1, 2022 Tomio Geron ( @tomiogeron ) is a San Francisco-based reporter covering fintech. He was previously a reporter and editor at The Wall Street Journal, covering venture capital and startups. Before that, he worked as a staff writer at Forbes, covering social media and venture capital, and also edited the Midas List of top tech investors. He has also worked at newspapers covering crime, courts, health and other topics. He can be reached at tgeron@protocol.com or tgeron@protonmail.com. August 1, 2022 Startup valuations have shot sky-high in recent years. Now they’re taking a breather — but there hasn’t been a way to bet on startups getting less valuable the way you can with publicly traded companies. That’s because shorting a company typically requires borrowing the stock, selling it, then buying it back at a lower price. That’s almost impossible to do, as private companies can often block sales. In the current down market for tech, the long-sought idea of shorting startups — a safe-seeming bet, since most startups fail — may be coming to fruition. Companies are offering synthetic derivatives and options so investors can take the equivalent of a short position. This isn’t about monetizing Silicon Valley schadenfreude, though. Solid financial reasons for wanting to short a startup range from public-market hedge funds hunting for alpha in private markets to VCs looking to hedge their own investments. Synthetic derivatives trade in concert with an underlying asset’s value, even though they are not directly tied to that asset. Synthetics first grew popular in other parts of the investing world; though they got a black eye in the 2007-2009 financial crisis, they’ve made something of a comeback in recent years. As Silicon Valley minted unicorn after unicorn, synthetics that let investors take long positions in startups without actually investing were sought-after alternatives for those shut out of hot deals. But providers of startup synthetics say the interest has strongly tilted to the short side recently. Selling interest increased to 80% of open order interest at Caplight, while the buy side has dropped to 20%, according to Javier Avalos, CEO of the private-market derivatives marketplace. It was previously fairly even or a 60/40 split, he said. “We basically saw the demand side of the Caplight marketplace go away for the past few months,” he said. Javier Avalos and Justin Moore co-founded Caplight, a private-market derivatives marketplace. Photo: Caplight Family offices, private equity, sovereign wealth and hedge funds previously took the long side of these bets, but many have dropped out recently, said Natalie Hwang, founder at Apeira Capital, which has a long and short investing strategy in private companies. The short interest, meanwhile, is still there, typically from hedge funds or public-market investors. Hwang calls it an opportunity to “capture negative value,” either to defend a portfolio or take advantage of volatility. The trade is risky — and involves complex investing strategies, and collateral is required. But this trade is being done largely by sophisticated institutional investors, Avalos said. The startup-synthetics sector has gone through several iterations. Two previous attempts at this were shut down by the SEC. In 2015, Sand Hill Exchange settled with the SEC for offering illegal derivatives to retail investors. And in 2016, Equidate, now Forge Global, settled SEC charges and agreed to shut down a service offering unregistered swaps on pre-IPO startups to provide liquidity for private company employees. Avalos said Caplight, which has a broker-dealer license, is different because it offers options, which are regulated differently than swaps, and because the options are offered only as private placements between institutional investors, typically with at least $100 million of assets. Apeira Capital founder Natalie Hwang. Photo: Apeira Capital Companies whose names are involved in the trades are not affected because their shares are not touched and the information rights and governance rights are not affected, Avalos said. Most of the investors now seeking to short startup stock are investors or shareholders who already have a sizable position in a company and don’t want to sell it, Avalos said. They see these derivatives as a hedge. Venture firms are among the groups doing this, taking some money off the table without having to actually sell a portion of their stake. This could be a growing strategy, especially for firms sitting on a large, aging stake in a company. Meanwhile, the companies generating most of the short interest are well-known private companies worth $10 billion or more that have a relatively strong amount of liquidity, often where investors are sitting on large gains, Avalos said. On the buy side, investors are often looking for earlier-stage companies worth $1 billion or more, he said. While the buy side has collapsed, some investors are starting to come back and place orders on certain targeted names that they’re interested in, Avalos said. He doesn’t expect the broader startup market to really take off until the prospects of IPOs return. In the meantime, options provide trading strategies that will bring in new investors who aren’t necessarily betting on the company’s long-term prospects. “The really cool part about introducing optionality like call and put options into this market is it gives you the opportunity to now invest without just needing the IPO window to be opened,” he said. “Because for really sophisticated institutional investors who are comfortable using options you can find really attractive option pricing, even if you're not necessarily making a bet on the fundamentals of those businesses.” That could make the startup market more like the traditional public markets — with all the speculative good and bad that goes with it. Keep ReadingShow less Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data. August 1, 2022 A quick-service restaurant chain is running its AI models on machines inside its stores to localize delivery logistics. At the same time, a global pharma company is training its machine learning models on premises, using servers it manages by itself. Cloud computing isn’t going anywhere, but some companies that use machine learning models and the tech vendors supplying the platforms to manage them say machine learning is having an on-premises moment. For many years, cloud providers have argued that the computing requirements for machine learning would be far too expensive and cumbersome to start up on their own, but the field is maturing. “We still have a ton of customers who want to go on a cloud migration, but we're definitely now seeing — at least in the past year or so — a lot more customers who want to repatriate workloads back onto on-premise because of cost,” said Thomas Robinson, vice president of strategic partnerships and corporate development at MLOps platform company Domino Data Lab. Cost is actually a big driver, said Robinson, noting the hefty price of running computationally intensive deep-learning models such as GPT-3 or other large-language transformer models, which businesses today use in their conversation AI tools and chatbots, on cloud servers. There's more of an equilibrium where they are now investing again in their hybrid infrastructure. The on-prem trend is growing among big box and grocery retailers that need to feed product, distribution and store-specific data into large machine learning models for inventory predictions, said Vijay Raghavendra, chief technology officer at SymphonyAI, which works with grocery chain Albertsons. Raghavendra left Walmart in 2020 after seven years with the company in senior engineering and merchant technology roles. “This happened after my time at Walmart. They went from having everything on-prem, to everything in the cloud when I was there. And now I think there's more of an equilibrium where they are now investing again in their hybrid infrastructure — on-prem infrastructure combined with the cloud,” Raghavendra told Protocol. “If you have the capability, it may make sense to stand up your own [ co-location data center ] and run those workloads in your own colo, because the costs of running it in the cloud does get quite expensive at certain scale.” Some companies are considering on-prem setups in the model building phase, when ML and deep-learning models are trained before they are released to operate in the wild. That process requires compute-heavy tuning and testing of large numbers of parameters or combinations of different model types and inputs using terabytes or petabytes of data. “The high cost of training is giving people some challenges,” said Danny Lange, vice president of AI and machine learning at gaming and automotive AI company Unity Technologies. The cost of training can run into millions of dollars, Lange said. “It’s a cost that a lot of companies are now looking at saying, can I bring my training in-house so that I have more control on the cost of training, because if you let engineers train on a bank of GPUs in a public cloud service, it can get very expensive, very quickly.” Companies shifting compute and data to their own physical servers located inside owned or leased co-located data centers tend to be on the cutting edge of AI or deep-learning use, Robinson said. “[They] are now saying, ‘Maybe I need to have a strategy where I can burst to the cloud for appropriate stuff. I can do, maybe, some initial research, but I can also attach an on-prem workload.” If you let engineers train on a bank of GPUs in a public cloud service, it can get very expensive, very quickly. Even though the customer has publicized its cloud-centric strategy, one pharmaceutical customer Domino Data Lab works with has purchased two Nvidia server clusters to manage compute-heavy image recognition models on-prem, Robinson said. High cost? How about bad broadband For some companies, a preference for running their own hardware is not just about training massive deep-learning models. Victor Thu, president at Datatron, said retailers or fast-food chains with area-specific machine learning models — used to localize delivery logistics or optimize store inventory — would rather run ML inference workloads in their own servers inside their stores, rather than passing data back and forth to run the models in the cloud. Some customers “don’t want it in the cloud at all,” Thu told Protocol. “Retail behavior in San Francisco can be very different from Los Angeles and San Diego for example,” he said, noting that Datatron has witnessed customers moving some ML operations to their own machines, especially those retailers with poor internet connectivity in certain locations. Model latency is a more commonly recognized reason to shift away from the cloud. Once a model is deployed, the amount of time it takes for it to pass data back and forth between cloud servers is a common factor in deciding to go in-house. Some companies also avoid the cloud to make sure models respond rapidly to fresh data when operating in a mobile device or inside a semi-autonomous vehicle. “Often the decision to operationalize a model on-prem or in the cloud has largely been a question of latency and security dictated by where the data is being generated or where the model results are being consumed,” Robinson said. Over the years, cloud providers have overcome early perceptions that their services were not secure enough for some customers, particularly those from highly regulated industries. As big-name companies such as Capital One have embraced the cloud , data security concerns have less currency nowadays. Still, data privacy and security does compel some companies to use on-prem systems. AiCure uses a hybrid approach in managing data and machine learning models for its app used by patients in clinical trials, said the company’s CEO Ed Ikeguchi. AiCure keeps processes involving sensitive, personally identifiable information (PII) under its own control. “We do much of our PII-type work locally,” Ikeguchi said. However, he said, when the company can use aggregated and anonymized data, “then all of the abstracted data will work with cloud.” Ikeguchi added, “Some of these cloud providers do have excellent infrastructure to support private data. That said, we do take a lot of precautions on our end as well, in terms of what ends up in the cloud.” “We have customers that are very security conscious,” said Biren Fondekar, vice president of customer experience and digital strategy at NetApp, whose customers from highly regulated financial services and health care industries run NetApp’s AI software in their own private data centers. Big cloud responds Even cloud giants are responding to the trend by subtly pushing their on-prem products for machine learning. AWS promoted its Outposts infrastructure for machine learning last year in a blog post , citing decreased latency and high data volume as two key reasons customers want to run ML outside the cloud. “One of the challenges customers are facing with performing inference in the cloud is the lack of real-time inference and/or security requirements preventing user data to be sent or stored in the cloud,” wrote Josh Coen, AWS senior solutions architect, and Mani Khanuja, artificial intelligence and machine learning specialist at AWS. In October, Google Cloud announced Google Distributed Cloud Edge to accommodate customer concerns about region-specific compliance, data sovereignty, low latency and local data processing. Microsoft Azure has introduced products to help customers take a hybrid approach to managing machine learning by validating and debugging models on local machines, then deploying them in the cloud. Snowflake , which is integrated with Domino Data Lab’s MLOps platform, is mulling more on-prem tools for customers, said Harsha Kapre, senior product manager at Snowflake. “I know we're thinking about it actively,” he told Protocol. Snowflake said in July that it would offer its external table data lake architecture — which can be used for machine learning data preparation — for use by customers on their own hardware . “I think in the early days, your data had to be in Snowflake. Now, if you start to look at it, your data doesn't actually have to be technically [in Snowflake],” Kapre said. “I think it’s probably a little early” to say more, he added. Hidden costs As companies integrate AI across their businesses, more and more people in an enterprise are using machine learning models, which can run up costs if they do it in the cloud, said Robinson. “Some of these models are now used by applications with so many users that the compute required skyrockets and it now becomes an economic necessity to run them on-prem,” he said. But some say the on-prem promise has hidden costs. “The cloud providers are really, really good at purchasing equipment and running it economically, so you are competing with people who really know how to run efficiently. If you want to bring your training in-house, it requires a lot of additional cost and expertise to do,” Lange said. Bob Friday, chief AI officer at communications and AI network company Juniper Networks, agreed. “It’s almost always cheaper to leave it at Google, AWS or Microsoft if you can,” Friday said, adding that if a company doesn’t have an edge use-case requiring split-second decision-making in a semi-autonomous vehicle, or handling large streaming video files, on-prem doesn’t make sense. But cost savings are there for enterprises with large AI initiatives, Robinson said. While companies with smaller AI operations may not realize cost benefits by going in-house, he said, “at scale, cloud infrastructure, particularly for GPUs and other AI-optimized hardware, is much more expensive,” he said, alluding to Domino Data Lab’s pharmaceutical client that invested in Nvidia clusters “because the cost and availability of GPUs was not palatable on AWS alone.” Everybody goes to the cloud, then they sort of try to move back a bit. I think it's about finding the right balance. Robinson added, “another thing to take into consideration is that AI-accelerated hardware is evolving very rapidly and cloud vendors have been slow in making it available to users.” In the end, like the shift toward multiple clouds and hybrid cloud strategies, the machine learning transition to incorporate on-prem infrastructure could be a sign of sophistication among businesses that have moved beyond merely dipping their toes in AI. “There's always been a bit of a pendulum effect going on,” Lange said. “Everybody goes to the cloud, then they sort of try to move back a bit. I think it's about finding the right balance.” Keep ReadingShow less