HOW TO MIGRATE FROM HORTONWORKS DATA PLATFORM TO CLOUDERA DATA PLATFORM?

In 2019 Cloudera presented a new platform – Cloudera Data Platform – as a universal solution that allows for data management in any environment: Public Cloud, bare metal, Private Cloud, and Hybrid Cloud.

According to the new development strategy, presented by Jan Kunigk, Cloudera operation CTO in EMEA, and Florian von Walter, chief manager of Cloudera engineering solutions in Germany, Austria, Eastern Europe and Russia – newspaper “Storage News” № 1 (76), 2020 – on-premise Hadoop based solutions development is the first stage, followed by migration to Public Cloud, and, finally, to Hybrid Cloud.

Following the new Cloudera strategy, it is recommended to migrate from platforms CDH – Cloudera Distribution of Hadoop – and HDP – Hortonworks Data Platform – to CDP, since CDH and HDP support ends on December 31, 2021, which means that respective products will not receive updates, and there will be no maintenance options available for them. Consequently, enterprise clients need to migrate from HDP/CDH to the actual stack, to maintain their solutions’ operability.

Why CDP?

HDP and CDH users are recommended to migrate to the actual Cloudera stack since Cloudera features the most complete set of tools for working with corporate data:

• Cloudera Data Platform – a platform for collecting and storing data, for building EDW, EDH.
• Edge & Flow Management – for end devices managing, controlling and monitoring.
• Streams Messaging – for real-time delivery of large volumes of data.
• Data Science Workbench – for data analysis and solving AI/ML problems.
• Cloudera Manager – cluster management subsystem.
• Cloudera also provides a full set of tools for solving Data Security, Data Management and Data Governance problems.
• Full-fledged technical maintenance is available for Cloudera solutions.

Compare functionality and components of the listed platforms.

In contrast to previous HDP, CDP is not available as an open free distribution, however, additional functionality and tools make Cloudera stack the most convenient and economically effective tool for building Hadoop-based enterprise solutions.

Getting ready for migration

Cloudera provides detailed instructions for organizing the migration process, featuring several scenarios:

Trial CDP versions are available for different environments:

• 48h Cloudera platform test-drive in the cloud
• a free trial version of CDP Private Cloud for familiarization and testing CDP Upgrade Advisor, which contains detailed recommendations per specific clusters, is also available

Integration process

1. Choose your CDP migration variant: either full upgrade or migration with fail-proof operation requirements.
2. Check upgrade requirements and satisfy all prior conditions.
3. Choose target environment:
   1. CDP Public Cloud is recommended by Cloudera for systems with up to 50 nodes:
      HDP to CDP Public Cloud
      CDH to CDP Public Cloud
   2. CDP Private Cloud is recommended by Cloudera for systems with more than 50 nodes:
      HDP to CDP Private Cloud
      CDH to CDP Private Cloud
   3. CDP on-premise is recommended for customers, who due to legislation or corporate policy, do not consider migration to the cloud.
4. Setup, migrate, test and confirm.

Migration plan example:

1. Migration of the dev environment from Hortonworks stack (HDP / HDF) to Cloudera (СDP / CDF) stack. 2 weeks
   1. Cleaning the test environment and preparing infrastructure and security requirements.
   2. Setting up and configuring the CDP dev environment.
   3. Transferring developments and data from the HDP/HDF dev environment to the CDP dev environment.
   4. Testing and tuning the CDP dev environment.
2. Розширення кластера TEST і міграція зі стека Hortonworks (HDP / HDF) на стек Cloudera (СDP / CDF). 2 weeks
   1. Cleaning the HDP/HDF dev environment.
   2. Setting up and configuring the CDP test environment.
   3. Transferring developments and data from the CDP dev environment to the CDP test environment.
   4. Testing and tuning the CDP test environment.
3. Building the prod cluster on Cloudera (СDP / CDF) stack. 3 weeks
   1. Cleaning the HDP/HDF prod environment.
   2. Setting up and configuring the CDP prod environment.
   3. Transferring developments and data from the HDP/HDF prod environment to the CDP prod environment.
   4. Testing and tuning the CDP prod environment.