In this post, I will explain the details on how to setup Databricks Workspace on AWS Cloud.
- Navigate to <https://www.databricks.com/try-databricks#account> and register a free trial account and select “AWS Cloud” as your compute cloud provider.
- Follow the instructions on https://docs.databricks.com/en/getting-started/community-edition.html
- Login to AWS Cloud Account with your AWS account credential.
- Return to the Databricks Console > Workspaces and click on “Start Quickstart” button. Click on the “allow pop up” icon on the “Safari” search bar to bring up AWS Console > CloudFormation service.
- Provide a name, i.e. “MLForProductionV1” for the workspace and click on continue.
- AWS Console will get populated with CloudFormation template which will create the necessary AWS resources such as S3 bucket, IAM roles and lambda function as screenshot below:
- Return to Databricks console and a new workspace is created.
- Click on the “Open” hyperlink to bring up the workspace.
- Click on the “Compute” service and create a computer cluster for the workspace:
- Single node
- Databricks Runtime Version
- ML > 11.3 LTS ML (includes Apache Spark 3.3.0, Scala 2.12)
- Node Type: i3.xlarge
