VAST Data partners with Vertica to enable large-scale real-time queries

We’re excited to bring Transform 2022 back in person on July 19 and pretty much July 20-28. Join AI and data leaders for insightful conversations and exciting networking opportunities. Register today!

The data lake is now a well-known concept and concept across the enterprise, as is the data warehouse.

But what about “data lakehouse?”

As the name suggests, the emerging architecture combines the functions of the data lake raw data repository with the reporting and analytics business intelligence (BI) of the data warehouse.

This architecture, sometimes referred to as the less catchy “unified analytics warehouse,” can manage an organization’s full complement of structured, semi-structured, and unstructured data. It can support many different data workloads and can be deployed on top of low-cost cloud storage systems.

“It comes down to insight,” said Jeff Denworth, co-founder and chief marketing officer of VAST Data, a flash data storage company. “It provides one comprehensive view of an entire data park.”

To provide companies with this vantage point and enable real-time queries on a large scale, VAST Data has partnered with database company Vertica. The partnership announced today unites VAST Data’s all-flash Universal Storage data platform with Vertica’s Eon Mode Architecture to create an all-flash data lakehouse. This helps enterprises consolidate their structured and unstructured data silos to democratize data for real-time data exploration, analysis and insights, Denworth said.

“Customers can start asking a lot more questions,” he said, “they can get answers to questions much faster.”

Modern data market: faster, stronger and better everywhere

The big data management market continues to grow as organizations collect data on an ever-increasing scale. Global Industry Analysts, Inc., has forecast it to reach $234.6 billion by 2026.

Equally growing is the list of companies that support data lakehouse architectures. These include big data giants Snowflake and Databricks, as well as Oracle Cloud Infrastructure (OCI) and Google, which launched its BigLake engine at its Cloud Data Summit earlier this month. Onehouse emerged from stealth in February with its open-source data lakehouse; Dremio recently raised $160 million in Series E and released a free edition of its SQL Lakehouse in March.

Databricks, founded in 2013 and reporting an estimated post-money valuation of $38 billion, has said 5,000 global organizations use its Databricks Lakehouse Platform. Snowflake’s offering, Upsolver, helps companies such as Peer39 with GDPR/CCPA compliance page-level information and ironSource to collect, store and prepare data to support multiple use cases.

Gerrit Kazmaier, vice president and general manager of Databases, Data Analytics and Business Intelligence at Google Cloud, said of his decision to enter the market: “Managing data across disparate lakes and warehouses creates silos and increases risk and cost, especially when data needs to be moved.”

Stacking up against the competition

With their partnership, VAST and Vertica aim to offer a unique offering in a growing number of competitors.

As Denworth noted, in the case of data, a huge problem for enterprises is compartmentalized storage. Historically, companies have built multiple data warehouses and multiple lakes of that data, resulting in silos. When they asked a question about data, he said, they didn’t necessarily receive relevant answers, or they faced response times that are very challenging based on the orientation of their data.

“If you want to ask a question and get the best possible answer, you really have to look at all the data as it comes into an organization,” Denworth said. “Historically, that was quite a challenge, because no system is designed to see essentially everything.”

The data lakehouse is thus designed to provide more insight from a broader view of data. This is an incredibly valuable tool, he said, for big data teams and data science organizations trying to be broader and more flexible with their data analytics.

“You don’t have to copy data from department to department to department now,” Denworth said. “You just create these stateless servers and they all have access to the same data underneath.”

For example, the new VAST-Vertica-enabled lake house is being used by the online travel agency Agoda in Singapore to support and improve the recommendation engine. A mobile casino gaming company also uses the architecture for its recommendation capabilities.

Typically, Denworth said, organizations think they need to move to the cloud to get the best multi-warehouse solution. Or, if they look on-premises, their options are systems that are large and slow, or small and “very expensive and very fast.”

“Flash is something that combines both: big, cheap and fast,” he said.

Customers moving to the VAST-Vertica Lakehouse model can save 80% to 90% while supporting capacity scale by a factor of 100, Denworth said. He stressed that the average data warehouse houses terabytes in factors of 10. But companies that move beyond the data warehouse model to the lakehouse model speak eloquently in petabyte terms.

“Our customers send huge, huge data sets to the system,” Denworth said.

VentureBeat’s mission is to be a digital city square for tech decision makers to learn about transformative business technology and transactions. Learn more about membership.

This post VAST Data partners with Vertica to enable large-scale real-time queries

was original published at “https://venturebeat.com/2022/04/19/vast-data-teams-up-with-vertica-to-enable-real-time-queries-at-scale/”