> ## Documentation Index > Fetch the complete documentation index at: https://docs.scanoss.com/llms.txt > Use this file to discover all available pages before exploring further. # Install Knowledge Base > Download and install the SCANOSS Knowledge Base or Test KB using the `kb-download.sh` script. The [`kb-download.sh`](https://github.com/scanoss/on-premise/blob/main/install/kb-download.sh) script provides a unified interface for retrieving the full Knowledge Base (KB), a KB update, the test KB, a SQLite KB snapshot or an HFH KB snapshot directly from the SCANOSS SFTP server. It connects to the server, displays the available versions and allows you to select the desired package for download. Before proceeding, it verifies that sufficient disk space is available. For KB updates, the downloaded package includes a separate import utility (`ldb-import.sh`) which is used to import the update into your local LDB instance. ## KB Download Prerequisites * **lftp** (recommended): Enables faster, parallel, and resumable downloads. You can install it using `apt install lftp`. If `lft`p is not available, the script can fall back to the standard `sftp` client. When running with the `-y` flag, the fallback to `sftp` is handeld automatically. * **sshpass** (required only for sftp fallback): Needed to provide the password non-interactively when using `sftp`. Install it with `apt install sshpass`. This dependency is not required when using `lftp`. * **SFTP credentials**: Supplied by the SCANOSS sales team, including host, port, username and password. ## Running the Script Make sure the script has execution permissions: ```bash theme={null} chmod +x install/kb-download.sh ``` Run the script without arguments for a fully interactive session: ```bash theme={null} ./kb-download.sh ``` ## Command-line Options All connection details, paths, and the download mode can be passed as arguments to skip the matching interactive prompt: ```bash theme={null} ./kb-download.sh -m update -h sftp.test.xyz -P 12345 -u USER21 -p mypassword ``` **Connection / mode** | Flag | Description | | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `-m` | Download mode: `full` (full KB), `update` (KB update), `test` (test KB), `sqlite` (SQLite KB snapshot), or `hfh` (HFH KB snapshot) | | `-h` | SFTP host | | `-P` | SFTP port | | `-u` | SFTP username | | `-p` | SFTP password | | `-t` | lftp parallel download threads (default: 25) | | `-d` | Force download tool: `lftp` or `sftp` | | `-C` | Enable SSH transport compression. Off by default. Useful on slow links; on fast links it typically *slows* the transfer down (zlib CPU exceeds the bandwidth saving for dense payloads like hashes and indexes). | **Path / version overrides** | Flag | Description | | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `-V` | KB version (full/update/sqlite/hfh mode); default: latest from `LATEST.txt` on the server | | `-o` | Destination path. Test mode: test-KB destination. Full mode: `oss` folder destination. (default: `/var/lib/ldb/oss`) | | `-r` | Full mode only: destination for non-oss items (default: `/tmp/scanoss_kb_full_`) | | `-D` | Update / sqlite / hfh mode: download base directory. Final path is `/` for update/sqlite and `/.zip` for hfh. Defaults: `/tmp/scanoss_kb_update` for update, current directory for sqlite and hfh | **Non-interactive flags** | Flag | Description | | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `-y` | Don't prompt; use defaults for unspecified values and auto-confirm the download. Required values (`-m`, `-h`, `-P`, `-u`, `-p`) must be supplied — the script exits with an error if any are missing. | | `-f` | Force download even if free disk space is below the required size. Only meaningful with `-y`; without `-y` you are prompted to confirm. | | `-?` | Show help | Any options not provided on the command line will be prompted interactively, unless `-y` is set. ## Non-interactive Examples ```bash theme={null} # Defaults (latest version, /var/lib/ldb/oss + /tmp/scanoss_kb_full_): ./kb-download.sh -y -m full -h sftp.test.xyz -P 12345 -u USER21 -p mypassword # Pin a version, custom paths, force on low disk space: ./kb-download.sh -y -f -m full -V 26.03 \ -o /data/oss -r /data/kb-rest \ -h sftp.test.xyz -P 12345 -u USER21 -p mypassword # Update mode with a custom download base directory: ./kb-download.sh -y -m update -D /opt/scanoss/updates \ -h sftp.test.xyz -P 12345 -u USER21 -p mypassword # SQLite KB into the current directory (final path: ./): ./kb-download.sh -y -m sqlite \ -h sftp.test.xyz -P 12345 -u USER21 -p mypassword # SQLite KB into a specific base directory: ./kb-download.sh -y -m sqlite -D /opt/scanoss/sqlite \ -h sftp.test.xyz -P 12345 -u USER21 -p mypassword # HFH KB (single .zip; lands at ./.zip): ./kb-download.sh -y -m hfh -h sftp.test.xyz -P 12345 -u USER21 -p mypassword # Test KB: ./kb-download.sh -y -m test -h sftp.test.xyz -P 12345 -u USER21 -p mypassword ``` ## How It Works 1. **Connection**: the script prompts for (or reads from arguments) the SFTP host, port, username, and password 2. **Mode selection**: you choose between downloading the full KB, a KB update, the test KB, or a SQLite KB snapshot, or an HFH snapshot 3. **Version discovery** (full/update/sqlite/hfh only): it connects to the SFTP server and lists all available versions of the chosen type, marking which one is the latest. Test KB has no versions — there is only one current test KB on the server. 4. **Selection** (full/update/sqlite/hfh only): you choose which version to download (defaults to the latest) 5. **Disk space check**: the script fetches the metadata and verifies there is enough free disk space before downloading. If not enough is available, you are warned and asked to confirm before continuing. (For HFH the size comes from `ls -l` on the remote `.zip` rather than `metadata.json`.) 6. **Download**: * **For full KB**: the download is split between two destinations. The `oss` folder (the main LDB data) goes to its own destination (default: `/var/lib/ldb/oss`), and all remaining files and folders go to a separate directory (default: `/tmp/scanoss_kb_full_`). Both destinations can be changed when prompted. * **For updates**: the whole update folder is downloaded to a single directory (default: `/tmp/scanoss_kb_update/`). * **For test KB**: the test KB's `oss` folder is downloaded to a single destination (default: `/var/lib/ldb/oss`). This will populate your LDB with the test data for verification. * **For SQLite KB**: the whole version folder is downloaded to `/` (default base: the current working directory). The folder contains one or more `.sqlite` files plus a `metadata.json`. The script downloads every file you have access to in that folder — different users may see different sets of `.sqlite` files depending on their SFTP permissions. * **For HFH KB**: a single `.zip` archive is downloaded to `/.zip` (default base: the current working directory). With lftp the file is fetched in parallel chunks (`pget -n `); with sftp it's a single-stream `get` with the live progress poller. The script does not extract the archive — leave that to your downstream tooling. 7. **Import (updates only)**: after downloading an update, run the `ldb-import.sh` script included in the downloaded folder ## Example Sessions ### Update ```bash theme={null} SCANOSS Knowledge Base Download ================================ Using lftp for downloads (parallel, resumable). What do you want to download? 1) Full KB 2) KB update 3) Test KB 4) SQLite KB 5) HFH KB Select [1-5]: 2 SFTP host: sftp.test.xyz SFTP port: 12345 SFTP username: USER21 SFTP password: Fetching available update versions... Available update versions: ------------------- 1) 26.01 2) 26.02 3) 26.03 (latest) ------------------- Select a update version to download [1-3] (default: 3): Selected update: 26.03 Download directory [/tmp/scanoss_kb_update]: Fetching update metadata... Update size: 5.2G Free space: 120G (on /dev/sda1) Disk space OK. lftp parallel threads [25]: Download update 26.03 to /tmp/scanoss_kb_update? [Y/n] Downloading with lftp (25 parallel threads, resumable)... Update downloaded to /tmp/scanoss_kb_update/26.03 Finished downloading update. Please run the ldb-import.sh script provided inside the update folder to import into the KB: /tmp/scanoss_kb_update/26.03/ldb-import.sh ``` ### Full KB ```bash theme={null} ./kb-download.sh -m full -h sftp.test.xyz -P 12345 -u USER21 SCANOSS Knowledge Base Download ================================ Using lftp for downloads (parallel, resumable). SFTP password: Fetching available full KB versions... Available full KB versions: ------------------- 1) 26.01 2) 26.02 3) 26.03 (latest) ------------------- Select a full KB version to download [1-3] (default: 3): Selected full KB: 26.03 Destination for 'oss' folder [/var/lib/ldb/oss]: Destination for remaining files/folders [/tmp/scanoss_kb_full_26.03]: Fetching full KB metadata... Full KB size: 210G Free space: 500G (on /dev/sda1) Disk space OK. lftp parallel threads [25]: Download full KB 26.03 to /var/lib/ldb/oss + /tmp/scanoss_kb_full_26.03? [Y/n] Downloading kb/full/26.03/oss with lftp (25 parallel threads, resumable)... Downloading kb/full/26.03/md5sum with lftp (25 parallel threads, resumable)... Downloading kb/full/26.03/26.03_files.md5sum with lftp (25 parallel threads, resumable)... Full KB downloaded: oss folder: /var/lib/ldb/oss other files: /tmp/scanoss_kb_full_26.03 Finished downloading full KB. ``` ### Test KB ```bash theme={null} ./kb-download.sh -m test -h sftp.test.xyz -P 12345 -u USER21 SCANOSS Knowledge Base Download ================================ Using lftp for downloads (parallel, resumable). SFTP password: Destination for test KB [/var/lib/ldb/oss]: Fetching test KB metadata... Test KB size: 180M Free space: 500G (on /dev/sda1) Disk space OK. lftp parallel threads [25]: Download test KB to /var/lib/ldb/oss? [Y/n] Downloading kb/test/oss with lftp (25 parallel threads, resumable)... Test KB downloaded to /var/lib/ldb/oss Finished downloading test KB. ``` ### SQLite KB ```bash theme={null} ./kb-download.sh -m sqlite -h sftp.test.xyz -P 12345 -u USER21 SCANOSS Knowledge Base Download ================================ Using lftp for downloads (parallel, resumable). SFTP password: Fetching available sqlite KB versions... Available sqlite KB versions: ------------------- 1) 26.04 (latest) ------------------- Select a sqlite KB version to download [1-1] (default: 1): Selected sqlite KB: 26.04 Download directory [.]: Fetching sqlite KB metadata... Sqlite KB size: 1.2G Free space: 500G (on /dev/sda1) Disk space OK. lftp parallel threads [25]: Download sqlite KB 26.04 to ./26.04? [Y/n] Downloading kb/sqlite/26.04 with lftp (25 parallel threads, resumable)... Sqlite KB downloaded to ./26.04 Finished downloading sqlite KB. ``` The downloaded folder contains all `.sqlite` files the SFTP user has access to (plus `metadata.json`). Different users may see different sets of files depending on their permissions. ## HFH KB ``` ./kb-download.sh -m hfh -h sftp.test.xyz -P 12345 -u USER21 SCANOSS Knowledge Base Download ================================ Using lftp for downloads (parallel, resumable). SFTP password: Fetching available hfh KB versions... Available hfh KB versions: ------------------- 1) 26.06 (latest) ------------------- Select a hfh KB version to download [1-1] (default: 1): Selected hfh KB: 26.06 Download directory [.]: Fetching hfh KB file size... Hfh KB size: 35G Free space: 1.7T (on /dev/sda1) Disk space OK. lftp parallel threads [25]: Download hfh KB 26.06 to ./26.06.zip? [Y/n] Downloading kb/hfh/26.06.zip with lftp (25 parallel chunks, resumable)... Hfh KB downloaded to ./26.06.zip Finished downloading hfh KB. ``` HFH ships as a single `.zip` archive (no `metadata.json`); the disk-space check reads the file size from the remote `ls -l` output. The script does not extract the archive — that's left to downstream tooling. ## Verifying a KB Update After importing an update, verify the installation by scanning the test WFP files included in the update directory: ```bash theme={null} scanoss-py scan /tmp/scanoss_kb_update/26.03/26.03_test_file.wfp scanoss-py scan /tmp/scanoss_kb_update/26.03/26.03_test_snippet.wfp ``` If both scans return appropriate matches, the update was imported successfully. > If the import is interrupted, it may cause corruption in the existing LDB. ## Next Steps Once the Knowledge Base installation is complete, proceed to [Verify Installation](verify-installation).