Return to site

AEM: Convert Blob Datastore to File Datastore

Typical AEM installations are based on TarMK and by default, blob data is stored within repository (referred as embedded blob store). File Datastore option in AEM provides the ability to store blob data in a file system outside the repository. The performance seems to be better with File Datastore, however I am unable to quantify the results as my local repo is relatively small. An embedded blob store can be converted to a File Datastore and the steps are documented below.

Step 1: Before starting, please take a backup of repository. I repeat, please take a backup of the repository :)

Step 2: Upgrade to latest service pack and oak. The latest service pack and oak version for your AEM version is listed on the version hot-fix page. Example: For 6.1, it is https://helpx.adobe.com/experience-manager/kb/aem61-available-hotfixes.html

Step 3: Shutdown your AEM instance

Step 4: Perform Compaction

Find the corresponding oak-run applicable for your repository. The oak-version of your repository should match the oak-run listed at https://repo1.maven.org/maven2/org/apache/jackrabbit/oak-run/

In case you missed taking a backup, please do it now:

  • Backup: java –jar oak-run-*.jar backup <aem-folder>/crx-quickstart/repository/segmentstore <backup>
  • Checkpoints: java -jar oak-run-*.jar checkpoints <aem-folder>/crx-quickstart/repository/segmentstore
  • Remove Unreference Checkpoints: java -jar oak-run-*.jar checkpoints <aem-folder>/crx-quickstart/repository/segmentstore rm-unreferenced
  • Compact: java -jar oak-run-*.jar compact <aem-folder>/crx-quickstart/repository/segmentstore

Step 5: Start AEM instance, Navigate to http://{host:port}/system/console/repositorycheck

Select Perform Traversal Check, Fix Inconsistencies, Data Store Consistency Check and execute "Run"

Step 6: Shutdown AEM instance

Step 7: Migrating Blob Store

I have seen references to online blogs to use oak-upgrade (https://github.com/apache/jackrabbit-oak) and I struggled a lot to identify which version actually works. Ideally, it should be the version of my oak but that did not work. After multiple attempts trying various oak versions, my final conclusion is it’s a hit or miss. Hence, the reason for stressing a lot on backups.

Adobe has a tool called crx2oak which helped to migrate the blob store to a custom file datastore successfully.

  • Download the latest crx2oak located at https://repo.adobe.com/nexus/content/groups/public/com/adobe/granite/crx2oak/
    • Note: I tried 1.4.6 and that worked without any issues on 6.1 SP2. As of this writing, there is a new version 1.6.8 and I would recommend trying to use the latest one.
  • Place the jar file in the same location as your quickstart jar file.
  • Create the directory for new datastore outside your crx-quickstart folder. Ex: /Users/sravan/datastore/ (make sure this directory has write permissions)
  • Create the directory for new repository inside your crx-quicskstart. Ex: crx-quickstart/repository_new
  • Run crx2oak using the command below

java -jar crx2oak-1.4.6-standalone.jar --copy-binaries --src-datastore=crx-quickstart/repository/segmentstore --datastore=/Users/sravan/datastore crx-quickstart/repository crx-quickstart/repository_new –mmap

  • Wait until the process completes. Do Not interrupt this process

Step 8: Configure AEM to use new repository and datastore

  • Remove crx-quickstart/repository and rename crx-quickstart/repository_new to crx-quickstart/repository
  • Create install directory inside crx-quickstart folder
  • Create two files with following entries
    • org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.config
      • path="/Users/sravan/datastore"
      • minRecordLength="256"
    • org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.config
      • customBlobStore=B"true"
  • Start AEM instance
All Posts
×

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!

OK