Install & Run PDP
ALPHA FEATURE - UNDER DEVELOPMENT
This documentation covers the PDP (Proof of Data Possession) feature, which is currently in alpha and under active development. This tool is intended for testing and experimental use only.
For production use and submitting real deals with live PDP Storage Providers, please use the Synapse SDK.
๐ Prerequisites
Note: This guide is written specifically for Ubuntu 22.04. If you are using a different Linux distribution, refer to the relevant documentation for package installation and compatibility.
Before starting, make sure you have a user with sudo privileges. This section prepares your system for the PDP stack.
โ๏ธ Hardware requirements
- RAM: 32 GiB+
- CPU: 8 Core+
- Storage:
- 1 TiB Fast storage (NVMe/SSD)
- 10 TiB Long-term storage (HDD)
- GPU: Not required
- Connectivity: Public HTTPS endpoint (domain)
๐งฐ System Package Installation
sudo apt update && sudo apt upgrade -y && sudo apt install -y \
mesa-opencl-icd ocl-icd-opencl-dev gcc git jq pkg-config curl clang \
build-essential hwloc libhwloc-dev libarchive-dev wget ntp python-is-python3 aria2
:hammer: Install Go (v1.24.0)
sudo rm -rf /usr/local/go
wget https://go.dev/dl/go1.24.0.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.24.0.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
source ~/.bashrc
go version
You should see something like: go version go1.23.7 linux/amd64
:wrench: Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
When prompted, choose the option 1) Proceed with standard installation (default โ just press Enter).
source $HOME/.cargo/env
rustc --version
You should see something like: rustc 1.86.0 (05f9846f8 2025-03-31)
๐ Add Go and Rust to Secure Sudo Path
sudo tee /etc/sudoers.d/dev-paths <<EOF
Defaults secure_path="/usr/local/go/bin:$HOME/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
EOF
โ๏ธ Installing and Running Lotus
๐ง Lotus is your gateway to the Filecoin network. It syncs the chain, manages wallets, and is required for Curio to interact with your node.
๐ง Build Lotus Daemon
Clone and check out Lotus:
git clone https://github.com/filecoin-project/lotus.git
cd lotus
git checkout $(curl -s https://api.github.com/repos/filecoin-project/lotus/releases/latest | jq -r .tag_name)
Build and Install for Mainnet
make clean && make lotus
sudo make install-daemon
lotus --version
Build and Install for Calibration
make clean && make GOFLAGS="-tags=calibnet" lotus
sudo make install-daemon
lotus --version
You should see something like: lotus version 1.32.2+calibnet+git.ff88d8269
๐ฆ Import a Snapshot and Start the Daemon
Download the Snapshot
Mainnet:
aria2c -x5 -o snapshot.car.zst https://forest-archive.chainsafe.dev/latest/mainnet/
Calibration:
aria2c -x5 -o snapshot.car.zst https://forest-archive.chainsafe.dev/latest/calibnet/
Import and Start the Daemon
lotus daemon --import-snapshot snapshot.car.zst --remove-existing-chain --halt-after-import
nohup lotus daemon > ~/lotus.log 2>&1 &
If you encounter errors related to EnableEthRPC or EnableIndexer, run the following command and restart Lotus
sed -i 's/^\( *\)#*EnableEthRPC = .*/\1EnableEthRPC = true/; s/^\( *\)#*EnableIndexer = .*/\1EnableIndexer = true/' ~/.lotus/config.toml
Monitor Sync Progress
lotus sync wait
To monitor continuously:
lotus sync wait --watch
Monitor Logs
tail -f ~/lotus.log
๐ Running YugabyteDB
๐ง Curio uses YugabyteDB to store metadata about deals, sealing operations, and PDP submissions.
๐ Set ulimit configuration
Before starting Yugabyte, you must increase the default ulimit values to ensure system limits do not interfere with the database.
To do this:
๐ Persist new limits across reboots
Add these lines to /etc/security/limits.conf:
echo "$(whoami) soft nofile 1048576" | sudo tee -a /etc/security/limits.conf
echo "$(whoami) hard nofile 1048576" | sudo tee -a /etc/security/limits.conf
This ensures the increased limits are automatically applied to future sessions.
โก Apply limit immediately (for current shell only)
ulimit -n 1048576
Verify:
ulimit -n
This should output 1048576.
โ๏ธ Install Yugabyte
wget https://software.yugabyte.com/releases/2.25.1.0/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz
tar xvfz yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz
cd yugabyte-2.25.1.0
./bin/post_install.sh
๐ Start the DB
./bin/yugabyted start \
--advertise_address 127.0.0.1 \
--master_flags rpc_bind_addresses=127.0.0.1 \
--tserver_flags rpc_bind_addresses=127.0.0.1
If you encounter locale-related errors when starting Yugabyte for the first time, run:
sudo locale-gen en_US.UTF-8
Visit http://127.0.0.1:15433 to confirm successful installation. This is the YugabyteDB web UI โ it should display the dashboard if the service is running correctly and all nodes are healthy.
You can also check your Yugabyte cluster details directly in the CLI with:
./bin/yugabyted status
๐งฑ Installing and Configuring Curio
๐ง Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs.
โ๏ธ System Configuration
Before you proceed with the installation, you should increase the UDP buffer size:
sudo sysctl -w net.core.rmem_max=2097152
sudo sysctl -w net.core.rmem_default=2097152
To make this change persistent across reboots:
echo 'net.core.rmem_max=2097152' | sudo tee -a /etc/sysctl.conf
echo 'net.core.rmem_default=2097152' | sudo tee -a /etc/sysctl.conf
๐ฌ Build Curio
Clone the repository and switch to the PDP branch:
git clone https://github.com/filecoin-project/curio.git
cd curio
git checkout pdpM3d
Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below.
Mainnet
make clean build
Calibration
make clean calibnet
This step will take a few minutes to complete.
โ Install and Verify Curio
Run the following to install the compiled binary:
sudo make install
This will place curio in /usr/local/bin
Verify the installation:
curio --version
Expected example output:
curio version 1.24.4+calibnet+git_f954c0a_2025-04-06T15:46:32-04:00
๐ง Guided Setup
Curio provides a utility to help you set up a new miner interactively. Run the following command:
curio guided-setup
1๏ธโฃ Select Curio Installation Type
Use the arrow keys to navigate the guided setup menu and select "Setup non-Storage Provider cluster".
2๏ธโฃ Enter Your YugabyteDB Connection Details
If you used the default installation steps from this guide, the following values should work:
- Host:
127.0.0.1 - Port:
5433 - Username:
yugabyte - Password:
yugabyte - Database:
yugabyte
You can verify these settings by running the following command from the Yugabyte directory:
./bin/yugabyted status
After selecting "Continue to connect and update schema", Curio will automatically create the required tables and schema in the database.
3๏ธโฃ Telemetry (Optional)
You'll be asked whether to share anonymised or signed telemetry with the Curio team to help improve the software.
Select your preference and continue.
4๏ธโฃ Save Database Configuration
At the final step of the guided setup, you'll be prompted to choose where to save your database configuration file.
Use the arrow keys to select a location. A common default is:
/home/your-username/curio.env
Once selected, setup will complete, and the miner configuration will be stored.
5๏ธโฃ Launch the Curio Web GUI
To explore the Curio interface visually, start the GUI layer:
curio run --layers=gui
Then, open your browser and go to:
http://127.0.0.1:4701
This will launch the Curio web GUI locally.
๐งช Enabling FWSS PDP
๐ง This section enables FWSS Proof of Data Possession (PDP) on your SP node using Curio. These steps guide you through running a standalone PDP service using Curio and pdptool.
| Cover image | ||
|---|---|---|
| PDP Support Channels | Filecoin.svg.png | https://filecoinproject.slack.com/archives/C0717TGU7V2 |
๐ฆ Attach Storage Locations
With Curio running with the GUI layer:
curio run --layers=gui
Run the following commands in your Curio CLI to attach storage paths:
curio cli storage attach --init --seal /fast-storage/path
curio cli storage attach --init --store /long-term-storage/path
Your fast-storage path should point to high-performance storage media such as NVMe or SSD
๐ง Add a PDP Configuration Layer
Browse to the Configurations page of the Curio GUI.
Create a new layer named pdp and enable the following under Subsystems:
You may find it helpful to search for the setting names in your browser.
- โ
EnableParkPiece - โ
EnablePDP - โ
EnableCommP - โ
EnableMoveStorage
In the HTTP section:
- โ
Enable:
true - ๐ DomainName:
your domain (e.g., pdp.mydomain.com) - ๐ก ListenAddress:
0.0.0.0:443
Tip: You must point your domain's A record to your server's public IP address for Let's Encrypt to issue a certificate.
๐ฐ Import your Filecoin Wallet Private Key:
There are several ways to obtain private keys for Ethereum addresses. In this guide, we will use a new delegated FIL wallet address.
Create a new delegated wallet:
lotus wallet new delegated
# Example output:
t410fuo4dghaeiqzokiqnxruzdr6e3cjktnxprrc56bi
You can display your Lotus wallets at any time by running:
lotus wallet list
Export & convert your new delegated wallet address private key:
lotus wallet export <your-delegated-wallet-address> | xxd -r -p | jq -r '.PrivateKey' | base64 -d | xxd -p -c 32
# Example output:
d4c2e3f9a716bb0e47fa91b2cf4a29870be3c5982fd6eafed71e8ac3f9c0b127
Browse to the PDP page of the Curio GUI and in the Owner Address section:
- Select Import Key
- Copy the previously generated private wallet key into the Private Key (Hex) field.
- Select Import Key
Your 0x wallet address - the delegated Ethereum address derived from your Filecoin delegated wallet private key - will be added to the Owner Address section of the Curio PDP page.
Make sure to send a small amount of FIL or tFIL (testnet FIL) to your 0x wallet - we recommend 8 FIL for Mainnet & 5 tFIL for Calibration to ensure uninterrupted PDP operation during initial setup and testing. Calibration test FIL faucet information.
Important: Secure your private key material. Don't expose or store it in plain text without protection.
๐ Restart and Verify
Restart Curio with both layers:
curio run --layers=gui,pdp
If you encounter errors related to EnableEthRPC or EnableIndexer, run the following command and restart Lotus
sed -i 's/^\( *\)#*EnableEthRPC = .*/\1EnableEthRPC = true/; s/^\( *\)#*EnableIndexer = .*/\1Enabl
If you encounter errors binding to port 443 when starting Curio with the pdp configuration layer, run:
sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/curio
Test the PDP service:
If pdptool is not installed, clone and build Curio:
git clone https://github.com/filecoin-project/curio.git
cd curio/cmd/pdptool
go build .
Generate a service secret:
./pdptool create-service-secret
./pdptool ping --service-url https://your-domain.com --service-name public
Always use public for the --service-name flag
Expected output:
Ping successful: Service is reachable and JWT token is valid.
๐ You're Done!
You've successfully launched a PDP-enabled Filecoin Storage Provider stack. Your system is now:
- โ Syncing with the Filecoin network via Lotus
- โ Recording deal and sector metadata in YugabyteDB
- โ Operating Curio to manage sealing and coordination
- โ Enabled Proof of Data Possession (PDP)
- โ Connected to your PDP-enabled storage provider
๐ Next Steps
- :heavy_check_mark: Register your FWSS node
- :link: Explore FWSS & PDP tools & resources at https://www.filecoin.services
- ๐ฌ Join the community - Filecoin Slack - #fil-pdp