Installation

Installation#

This section provides detailed instructions for installing the software and its dependencies.

Prerequisites#

You need to have both Redis and Kvrocks built from source and available locally. By default, the standard scripts of Vulnerability-Lookup expect the following directory structure:

/vulnerability-lookup   # Vulnerability-Lookup codebase
/redis
/kvrocks

Specifically:

  • ./vulnerability-lookup/cache/run_redis.sh runs ./redis/src/redis-server

  • ./vulnerability-lookup/storage/run_kvrocks.sh runs ./kvrocks/build/kvrocks (or starts the apache/kvrocks Docker container)

The Vulnerability-Lookup launcher provides configuration for both Redis and Kvrocks.

Redis#

Redis is an open-source (BSD-licensed) in-memory data structure store used as a database, cache, and message broker.

Note

Clone and build Redis in the same parent directory as this repository — not inside it.

Install required packages:

sudo apt-get update
sudo apt install build-essential tcl

Then build Redis:

git clone https://github.com/redis/redis.git
cd redis
git checkout 7.2
make
# Optionally, run the tests:
make test
cd ..

Kvrocks#

Kvrocks is a distributed NoSQL key-value database built on RocksDB and compatible with the Redis protocol. Its goal is to reduce memory usage while maintaining Redis compatibility.

Note

Kvrocks should be installed from the source, and the repository must be in one directory up as the one you will be cloning vulnerability-lookup into. It is also possible to make the apache/kvrocks docker container available to use.

Install the required packages:

sudo apt-get update
sudo apt install git gcc g++ make cmake autoconf automake libtool python3 libssl-dev

Then clone and build Kvrocks:

git clone --recursive https://github.com/apache/kvrocks.git
cd kvrocks
git checkout v2.13.0
./x.py build
cd ..

Import CSAF Sources#

  1. Build the CSAF support tools. (Requires Go ≥ 1.23)

  2. Verify the downloader exists and works:

$ ./bin-linux-amd64/csaf_downloader -h
Usage:
  csaf_downloader [OPTIONS] domain...
# (full usage instructions omitted for brevity)

  1. Add the full path to the downloader binary in config/generic.json under the key csaf_downloader_path.

Source Code#

Clone Vulnerability-Lookup with its submodules:

git clone --recursive https://github.com/vulnerability-lookup/vulnerability-lookup.git

Dependencies#

From within the cloned directory, install dependencies:

poetry install

Initialize the environment file:

echo VULNERABILITYLOOKUP_HOME="$(pwd)" >> .env

Note

VULNERABILITYLOOKUP_HOME must match the name defined in vulnerability-lookup/default/__init__.py.

Initialize submodules (this may take a while):

git submodule update --init

Configuration#

Generic Configuration#

Copy the sample configuration files and adjust them to your needs:

cp config/generic.json.sample config/generic.json
cp config/logging.json.sample config/logging.json
cp config/website.py.sample config/website.py
cp config/stream.json.sample config/stream.json

Modules#

Request your API keys for NVD and VARIoT:

Then copy and edit the module configuration file:

cp config/modules.cfg.sample config/modules.cfg

You can enable or disable individual feeders by setting enabled = false in their respective configuration sections. By default, all feeders are enabled.

User Accounts#

Enable user accounts, comments, and bundles by setting "user_accounts": true in config/generic.json.

Create a PostgreSQL user and database:

sudo apt install postgresql
sudo -u postgres createuser -P <username>
Enter password for new role: <password>
sudo -u postgres createdb <database> -O <username>

Update config/website.py with the correct credentials under DB_CONFIG_DICT, and set DATABASE_NAME (default: vulnlookup).

Initialize the database:

poetry run flask --app website.app db_init             # initialize database
poetry run flask --app website.app db stamp head       # create alembic_version table

Create the first admin user:

poetry run flask --app website.app create_admin --login admin --email admin@example.org --password <password>

Optionally, import OSI licenses and programming languages:

poetry run flask --app website.app import_osi_approved_licenses
poetry run flask --app website.app import_languages

Initialize the local GCVE registry:

poetry run flask --app website.app update_gcve_registry

Usage#

Start the application (from the project directory):

poetry run start

Note

On first launch, the system will download and process the complete archive from all feeds. This may require substantial CPU time. Subsequent runs will be much lighter.

Stop the application:

poetry run stop

By default, the web interface is available at: http://0.0.0.0:10001

Use the web UI or API to explore and interact with the system.

To update the tool:

poetry run update

Launching the Website with systemd#

Note

This is an alternative startup method. The website is automatically launched with the start command.

Create the file /etc/systemd/system/vulnerability-lookup-web.service:

[Unit]
Description=Vulnerability-Lookup Web service
After=network.target

[Service]
Type=forking
User=<system user used to install the project>
Group=<system user used to install the project>
WorkingDirectory=<path to the directory where you cloned the repository>
Environment="PATH=<path-to-virtualenv>/bin:/home/<user>/.local/bin:/usr/bin"
Environment="VULNERABILITYLOOKUP_HOME=<path to the directory where you cloned the repository>"
ExecStart=/bin/bash -c "exec poetry run start"
ExecStop=/bin/bash -c "exec poetry run stop"
StandardOutput=append:/var/log/vulnerability-lookup_message.log
StandardError=append:/var/log/vulnerability-lookup_error.log

[Install]
WantedBy=multi-user.target

Then reload and enable the service:

sudo systemctl daemon-reload
sudo systemctl enable vulnerability-lookup-web.service
sudo systemctl start vulnerability-lookup-web.service
systemctl status vulnerability-lookup-web.service

To follow the service logs live:

sudo journalctl -u vulnerability-lookup-web.service -f
Contents