Sudharsan Aravamuthan, APAC Solution Lead for Content and Data Intelligence solutions, Hitachi Vantara
Never has the need been more urgent to address the significance of object storage than it is today. The unprecedented pace and scale of advances in technology have transformed business landscapes all over the globe, and, in turn, triggered changes in requirements and processes, approach and practice with the explosion in data volumes. According to IDC, approximately only 2.5 per cent of all data is analysed. The remainder is lost, and its potential value squandered. IDC also estimates that 80 per cent of the data generated worldwide will be unstructured by 2025.
Modern data platforms need to enable the easy management of volumes of data and allow a seamless response to application demands and wide data accessibility, while satisfying compliance requirements.
Object storage goes mainstream
Since it was first introduced, over three decades ago, the concept of object storage has evolved into a mainstream technology of choice for addressing unstructured and semi-structured data and has become the cornerstone for its storage, protection and management. In the 1990s, first-generation object storage solutions were focused on archiving and protecting data, largely from a compliance perspective. The focus moved, in the second generation, to hybrid cloud, metadata enrichment, digital workplace enablement, ransomware protection and so forth. Today, object storage is catering to almost all tier-1 workloads.
With digital transformation having accelerated the adoption of cloud, social media, mobility, Internet of Things (IoT) and big data, many new data sources and modern application workloads have emerged and are driving object-storage consumption. Applications such as ultra-high-performance artificial intelligence and machine learning (AI/ML) workloads, IoT and unified data warehouses are leading this race.
In the current era of analytics, organisations seek rapid data-focused insights that can help them make informed business decisions. A high-performance object storage solution can enable additional modern and cloud-native applications and help organisations with its scalability, fast data retrieval and cost effectiveness.
Object storage and performance
Traditionally, organisations have turned to object storage for highly scalable, cost-efficient, long-term storage with easy data retrieval — but not for performance. The setting is different today and organisations need to use all their data, rather than have it stored in silos. Valuable insights are available from data lakes, content repositories and various archives from applications that organisations can leverage for better business results. With modern object storage use cases, the demand for scale and performance has never been greater. New solutions are required to deliver intelligent and dynamic data services to enable digital transformation.
Object storage of the past was synonymous with a low performance and high-capacity archival solution. Modern object stores, while supporting traditional applications with economical and massively scalable storage, have transformed into storage solutions for high-speed, tier-1 workloads. Object storage can be deployed for tier-1 workloads and help transform operations. A company whose primary use case is streaming data for log analytics was able to deliver faster, more accurate data insights across multiple data sources with cost efficiency. It used an object storage solution that helped deliver 1TB/minute throughput for a multi-petabyte Hadoop data lake using the S3A protocol. The company now expects to grow fourfold in two years.
Object storage solutions also consolidate high performance as a financial services organisation was able to successfully consolidate regulated unstructured data from content management and data repositories, taking data from high-speed fibre channel storage and NAS platforms. Using multiple all-flash object clusters, this company consolidated multiple service tiers into a single, high-performance tier.
Next-generation object stores build on insights shaped by years of experience and customer feedback from thousands of enterprise professionals actively engaged in managing corporate data storage needs. As a consequence, today organisations are witnessing a transition in both technology and business models that give IT organisations tremendous agility in deciding how they will operate in the future. With the advent of IoT and sophisticated business analytics, organisations are prioritising scale, performance, hybrid cloud capabilities and investment protection, which features software-defined microservice architecture built for limitless scale and deployment flexibility.
The future of object storage
A key part of the strategy is to provide a centralised governance hub capable of federating data from multiple disparate sources. Through this federated hub, organisations can efficiently collect metadata from all sources and use it to decide and act in accordance with corporate policy objectives. This independently scalable metadata repository makes securing and governing data from multiple sources possible. Other foundational capabilities include a modernised application programming interface (API) and execution model, scale-out policy engines, and an execution and event notification framework that enables seamless integration with third party on-premises or public cloud storage services.
In addition, next-generation object stores allow organisations to add all the custom metadata they need. They also support workloads, from real-time log aggregation to IoT, and transaction logs of high-performance databases. With enterprises facing new revenue-generating opportunities and workloads reliant on unstructured data, it clearly is time to maximise infrastructure advantage with object storage. Object storage can redefine the purpose and value of data, as well as significantly enhance data management and governance, while drastically lowering storage costs.
Today, cutting-edge object storage solutions exist, with built-in data security and protection, advanced metadata-based intelligence and modern hybrid cloud workflow capabilities. They are designed from the ground up to address the growing requirements and broader benefits of software-defined storage. They enable high performance and massive scalability to support hundreds of nodes and billions of objects and eliminate the classic database and network bottlenecks that plague distributed system designs. A modern object store is viewed as a solution with built-in data security and protection, advanced metadata-based intelligence and modern hybrid cloud workflow capabilities. It is the ideal foundation for intelligent and dynamic data services that organisations need to power their digital transformation journey.
