✌️How to use HPS in Japan region (DDN)?

Initial Setup

To start using File Storage - High Performance (HPS), please get in touch with the FPT Support team. After providing the required information, your account will be activated with the service, a quota will be assigned.

Required Information

Item

Unit

Required

Description

Tenant/ Region

—

Yes

The Tenant and Region where you want to enable the File Storage - High Performance service.

Storage Quota

Yes

The amount of storage capacity you plan to use.

Subnet

Subnet CIDR

Yes

Please provide the list of subnets that require access so the FPT support team can configure them.

Performance

IOPS

Desired IOPS performance (if not provided, the default limit will be applied).

Bandwidth

Gbps

Desired bandwidth (if not provided, the default limit will be applied).

Use with Metal Cloud (a Bare Metal GPU server)

Notes:

Each subnet has its own endpoint for connecting to the file storage system.
Always verify the endpoint and configuration details provided before performing the mount operation to avoid setup errors.

Use FPT Images

If you are using an image provided by FPT, the DDN client is pre-installed. Follow the steps below to mount the File Storage.

Create a Mount Point

# Create Mount Point
 
mkdir /mnt/hps

Mount the File System manually

# Create Mount Point
 
mount -t lustre 100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch /mnt/hps

Auto-Mount Configuration

To automatically mount the Lustre file system after reboot, add the following entry to /etc/fstab:

## LustreFS ##
100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch /mnt/hps lustre flock,defaults 0 0

Explanation

100.69.252.1@o2ib,100.69.252.2@o2ib:...:100.69.252.8@o2ib:/scratch
- 100.69.252.1–100.69.252.8: Fixed endpoint addresses of the Lustre servers.
- /scratch: Filesystem path on the Lustre server.
/mnt/hps: Mount point on the client (created in Step 1).

Note: Run the following command to verify that the mount configuration has no errors.

mount -a

Verify

# Confirm Client can mount lustre
df -h -t lustre
 
 
Filesystem                                                                       Size  Used Avail Use% Mounted on
100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch   11T  8.0K   11T   1% /mnt/hps

Using Custom Images (Ubuntu)

If you are using a custom image running Ubuntu, you will need to manually build the required packages by following the steps below.

Build Lustre Client

This step is for building the Lustre Client package from source code, in case a prebuilt package compatible with your current kernel version is not available.

If your system already has a Lustre Client package compatible with the current kernel, you can skip this step and proceed directly to Step B2 (Install .deb packages).
If you have previously built the package for the same kernel version, you can reuse it and skip this step

# Build Packages (skip if already built for same kernel)
 
wget https://s3-sgn10.fptcloud.com/file-storage-resource/exa-client-6.3.4.tar.gz
tar xzvf exa-client-6.3.4.tar.gz
cd exa-client
./exa_client_deploy.py
 
# Select option 2
# Wait for configure and build
# After installation is finished, select option 6
# (compiled packages are saved in /opt/ddn/exascaler/debs)

If your Ubuntu operating system is running kernel version 5.15.0-130-generic, you can skip the build process and directly use the prebuilt packages provided below.

# Install Lustre Pkg (for kernel 5.15.0-130-generic)
 
wget https://s3-sgn10.fptcloud.com/package-pub/lustre-dev_2.14.0-ddn184-1_amd64.deb
wget https://s3-sgn10.fptcloud.com/package-pub/lustre-client-modules-5.15.0-130-generic_2.14.0-ddn184-1_amd64.deb
wget https://s3-sgn10.fptcloud.com/package-pub/lustre-client-utils_2.14.0-ddn184-1_amd64.deb

Install Lustre Client

Install the Lustre Client using .deb packages built in step 1 or downloaded above.

Only perform this step if the matching .deb packages are available. If not, return to Step 1 to build the required packages.
If the kernel version does not match the package, errors may occur, and a rebuild will be necessary

# Install Lustre Pkg (for kernel 5.15.0-130-generic)
 
sudo dpkg -i lustre-dev_2.14.0-ddn184-1_amd64.deb \
             lustre-client-modules-5.15.0-130-generic_2.14.0-ddn184-1_amd64.deb \
             lustre-client-utils_2.14.0-ddn184-1_amd64.deb

Configure the Lustre Client

Apply the following configuration settings to optimize the performance of the Lustre Client.

Edit /etc/modprobe.d/lustre.conf:

# Edit or create file /etc/modprobe.d/lustre.conf like below
[/etc/modprobe.d/lustre.conf]
options lnet networks=o2ib(bond0)    <==HERE!
options ko2iblnd peer_credits=32 peer_credits_hiw=16 concurrent_sends=64

Edit /etc/lustre_client:

# Edit or create file /etc/lustre_client like below
[/etc/lustre_client]
 
#++++++++++++++++++++++++++++++++++++
# LNET Interface
IF1=bond0        <==HERE!
IF2=none
IF3=none
IF4=none
IF5=none
IF6=none
IF7=none
IF8=none

Create a mountpoint in the OS

# Create Mount Point
 
mkdir /mnt/hps

Mount the File System manually

# Create Mount Point
 
mount -t lustre 100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch /mnt/hps

Auto-Mount Configuration

To automatically mount the Lustre file system after reboot, add the following entry to /etc/fstab:

# Write followings entries to /etc/fstab
##LustreFS##
 
100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch /mnt/hps lustre flock,defaults 0 0

Explanation

100.69.252.1@o2ib,100.69.252.2@o2ib:...:100.69.252.8@o2ib:/scratch
- 100.69.252.1–100.69.252.8: Fixed endpoint addresses of the Lustre servers.
- /scratch: Filesystem path on the Lustre server.
/mnt/hps: Mount point on the client (created in Step 4).

Note: Run the following command to verify that the mount configuration has no errors.

mount -a

Verify

# Confirm Client can mount lustre
df -h -t lustre
 
 
Filesystem                                                                       Size  Used Avail Use% Mounted on
100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch   11T  8.0K   11T   1% /mnt/hps

Use with a GPU Virtual machine

Attach Network Interface and Configure IP Address

To connect to HPS Storage for H200, use the pre-created subnet named vm-gpu-hps-net-xxx located under Network > Subnets.

Attach the Network Interface:

Navigate to Instance Management
Select your target VM GPU
Open the Network Interface tab
Click Create NIC
Select the network named vm-gpu-hps-net-xxx

Configure the Network:

After attaching the interface, access your server and configure the network settings manually if DHCP is not enabled for that interface.

Build and Install Lustre Client

This step is for building and installing the Lustre Client package from source code

# Build Packages (skip if already built for same kernel)
 
wget https://s3-sgn10.fptcloud.com/file-storage-resource/exa-client-6.3.4.tar.gz
tar xzvf exa-client-6.3.4.tar.gz
cd exa-client
./exa_client_deploy.py
 
# Select option 2
# Wait for configure and build
# After installation is finished, select option 6
# (compiled packages are saved in /opt/ddn/exascaler/debs)

Configure the Lustre Client

Apply the following configuration settings to optimize the performance of the Lustre Client.

Edit /etc/modprobe.d/lustre.conf:

# Edit or create file /etc/modprobe.d/lustre.conf like below
[/etc/modprobe.d/lustre.conf]
options lnet networks=tcp(eth1)    <==HERE!
options ko2iblnd peer_credits=32 peer_credits_hiw=16 concurrent_sends=64

Edit /etc/lustre_client:

# Edit or create file /etc/lustre_client like below
[/etc/lustre_client]
 
#++++++++++++++++++++++++++++++++++++
# LNET Interface
IF1=eth1        <==HERE!
IF2=none
IF3=none
IF4=none
IF5=none
IF6=none
IF7=none
IF8=none

Create a mountpoint in the OS

# Create Mount Point
 
mkdir /mnt/hps

Mount the File System manually

# Create Mount Point
 
mount -t lustre 100.69.252.1@tcp,100.69.252.2@tcp:100.69.252.3@tcp,100.69.252.4@tcp:100.69.252.5@tcp,100.69.252.6@tcp:100.69.252.7@tcp,100.69.252.8@tcp:/scratch /mnt/hps

Auto-Mount Configuration

To automatically mount the Lustre file system after reboot, add the following entry to /etc/fstab:

# Write followings entries to /etc/fstab
##LustreFS##
 
100.69.252.1@tcp,100.69.252.2@tcp:100.69.252.3@tcp,100.69.252.4@tcp:100.69.252.5@tcp,100.69.252.6@tcp:100.69.252.7@tcp,100.69.252.8@tcp:/scratch /mnt/hps lustre flock,defaults 0 0

Explanation

100.69.252.1@tcp,100.69.252.2@tcp:...:100.69.252.8@tcp:/scratch

100.69.252.1–100.69.252.8: Fixed endpoint addresses of the Lustre servers.
/scratch: Filesystem path on the Lustre server.

/mnt/hps: Mount point on the client (created in Step 4).

Note: Run the command to verify if the mount configuration has any errors.

mount -a

Verify

# Confirm Client can mount lustre
df -h -t lustre
 
 
Filesystem                                                                       Size  Used Avail Use% Mounted on
100.69.252.1@tcp,100.69.252.2@tcp:100.69.252.3@tcp,100.69.252.4@tcp:100.69.252.5@tcp,100.69.252.6@tcp:100.69.252.7@tcp,100.69.252.8@tcp:/scratch   11T  8.0K   11T   1% /mnt/hps

PreviousHow to use HPS in Vietnam region (VAST Data)?NextBefore you start

Last updated 1 month ago

Was this helpful?

hashtagInitial Setup

hashtagUse with Metal Cloud (a Bare Metal GPU server)

hashtagUse FPT Images

hashtagUsing Custom Images (Ubuntu)

hashtagUse with a GPU Virtual machine

hashtagAttach Network Interface and Configure IP Address

Initial Setup

Use with Metal Cloud (a Bare Metal GPU server)

Use FPT Images

Using Custom Images (Ubuntu)

Use with a GPU Virtual machine

Attach Network Interface and Configure IP Address