Solutions Review’s Expert Insights series is a collection of contributed articles written by industry experts in the enterprise software category. In this feature, Dataddo CEO Petr Nemeth reveals the most important big data trends and predictions for 2023.
Contrary to expectations, the percentage of companies investing in digital transformation today is not as high as it was before the pandemic.what teeth Today, however, a higher proportion of companies are in a more advanced transformational stage.
They are using more data-producing tools, sharing data with more end users, and making more collaborative efforts to manage their data.
This raises many questions about effective data management and the future of BI. Is the need for more tools endless? How do you ensure the data they generate is continuously integrated, shared, and correctly interpreted? How do you keep your data safe and clean? What should I do now?
Below are seven predictions that will help business leaders predict the answers to these questions in 2023 and beyond.
Big Data Trends and Predictions for 2023
Diversification of data creation tools, but shorter customer lifecycles for each tool
There is no doubt that the number of SaaS tools available and the amount of data they collectively generate will continue to grow. Look at the size of the SaaS market in 2023. It is predicted to be worth twice as much as it was in 2019. Companies are adopting more tools each year and there is no clear end in sight.
But one of the less obvious side effects of this is that the average customer lifecycle of these tools can be shortened.
Millions of dollars are wasted annually on little-used tools, big and small. They are always trying new things and forgetting other things at the same time.
Additionally, because many of these tools have been adopted at the department, team, and employee level, large enterprises are unaware of about half of the deployed SaaS tools, while small businesses are aware of about a third. I have not.
IT departments will increase consolidation and purging to combat the pile of unused tools. This, coupled with increased adoption, will shorten the lifecycle of most SaaS tools.
Exceptions are tools that are integral to a company’s infrastructure, such as CRM and data integration tools.
Architecture-agnostic data integration
It’s common for companies today to use separate platforms for ETL/ELT, reverse ETL, and sometimes data replication.
ETL/ELT and data replication are well-established processes in the world of data integration, but reverse ETL is a very new process and only offered by a few specialized vendors, so this makes sense.
Since Reverse ETL is also the final piece of modern data architecture, companies interested in it typically already have established relationships with vendors of ETL/ELT and data replication solutions. So it might seem logical to look for another platform specifically for reverse ETL.
However, over time, data integration will become such a central aspect of business that companies will no longer recognize the difference in integration processes. Tools for integration are easier to use and users don’t have to think about the kind of engineering that connects data sources and data destinations.
Instead, you need one architecture-agnostic platform for all integration types. Select Source, Destination and Destination.
Business professionals will become more data literate and low-code to no-code BI, and data integration tools will become the norm
A high proportion of non-technical professionals recognize the need for data savvy (58% according to a 2022 survey by Qlik), and an even higher proportion of decision makers expect to be data savvy 82 (according to Tableau 2022 study by Forrester). If these professionals want to remain relevant in the job market, they will need to develop competencies that were once the domain of engineers only.
Fortunately for them, less and less technical knowledge is required to work with data tools (BI tools, data integration tools, and even some data storage).
Gartner predicts that by 2025, 70% of new applications developed by enterprises will rely on low-code or no-code technologies. The terms “low-code” and “no-code” are often used to describe development platforms, but are increasingly used to describe BI and data integration platforms.
This trend, combined with the push for data literacy within the enterprise, effectively offloads the heavy lifting from engineers and allows non-technical employees to build their own data solutions.
Demand for Citizen Data Scientists Continues
A citizen data scientist is a business sector expert who has some knowledge of data and analytics, and possibly coding, but is not a full-fledged data scientist. In the near future, they will play a key role in bridging the gap between business and data teams. Their duties range from determining success measures, collecting and interpreting data, to evaluating and deploying data models.
The U.S. Bureau of Labor Statistics predicts that by 2029, the field of data science will grow more than any other field. So it’s no wonder global companies like BP and Epsilon are already benefiting from citizen data scientists.
The rise of this new class of experts will have a decentralized impact on the data governance policies of many enterprises, as defined by the hub-and-spoke governance model.
As a result, business teams are empowered and data teams’ focus shifts to security and quality.
Data security becomes a major concern for buyers
Decentralization of data competencies is necessary for organizations that need greater flexibility in analytics at the operational level. But as data breaches and other privacy issues become more common, they are also exposed to a higher degree of risk.
In Europe, data protection authorities have consistently imposed fines for GDPR violations, with the toughest fines going to technology companies. The highest payout so far in 2022 was €405 million (or $402 million) for him, which was slapped on Instagram owner Meta Platforms Ireland Limited in September.
In the United States, there are no federal data privacy laws, but businesses should pay attention to state laws. And, of course, hackers. Just this year Microsoft, Uber, Red Cross and News Corp were all hacked.
SaaS buyers are starting to notice and will become more conscious of the data they provide to vendors. Vendors will find it difficult to close large deals without a certification like SOC 2. Dataddo lets you see this firsthand. Ultimately, data security trumps other purchasing criteria such as ease of use and price.
Data quality remains a challenge and AI will play a bigger role in cleaning data
Ever since people started collecting data, data quality has been a challenge. But today’s data comes from more and more disparate sources and is processed by more and more line-of-business professionals, making the costs of mistakes spreading to downstream systems more tangible.
Gartner estimates that in 2021, organizations will incur an average of $12.9 million in malicious data costs annually.
Data quality will never be perfect, but one of the big contributors to keeping it high is the incremental implementation of AI-based mechanisms in analytics and data integration tools. (Dataddo, for example, is an integrated tool with AI anomaly detection under the hood.)
These technologies are getting better at flagging outliers and keeping missing, inaccurate, and corrupt data out of pipelines and dashboards.
It is also important to note that AI-based data quality solutions are always most effective when analyzing large datasets over time and should be implemented alongside traditional human-centric solutions. am.
BI tools become mobile-friendly for passive use
BI’s entry into the mobile space seems natural.
Marketers, salespeople, executives, and other regular users of data often need to access it when they are not in front of their computer. And professionals who don’t spend most of their day in front of a computer, such as warehouse workers and truck drivers, are starting to need more regular computer access.
Therefore, it should come as no surprise that the market value of mobile BI is expected to rise from $10 billion in 2021 to around $55.5 billion in 2030. In 2020 he will increase from $35.2 billion to $224.2 billion in 2028.
This supports the prediction that mobile BI tools, no matter how sophisticated and streamlined, will primarily serve the following purposes: delivery insight.for manufacturing For example, for deep drill-downs and heavy dashboard customization insights, the desktop interface will always be king.
one step ahead
The race to digital transformation is highly dynamic. But one way to stay ahead is to pay attention to emerging trends in data management and BI. They can give you a sneak peek at what’s around the corner and help inform the strategies you implement today.
Businesses should consider:
- Proactively drive adoption of SaaS tools (and reduce pile-up) by providing more support to end users.
- Invest in future-proof data integration tools.
- Developing data literacy in non-technical business professionals.
- We make every effort to comply with and remain compliant with international data security standards.
These are all great opportunities, and now is the perfect time.
Petr Nemeth is the founder and CEO of Dataddo. Dataddo is a code-free integration platform that connects cloud-based applications with his BI tools, data warehouses, and data lakes. Prior to founding Dataddo, Petr worked as a developer, analyst and systems architect for telecoms, IT and media companies, working on large scale projects involving the Internet of Things, Big Data and Business Intelligence. rice field.
