feat: move scanpipe settings to local settings.py with lazy resolutio…

[tdruez]

…n and SCANPIPE dict override

Issues

• Issue: scanpipe as stand-alone library #2179

Changes

Move all scanpipe app-level settings out of scancodeio/settings.py into a new scanpipe/settings.py module. This makes scanpipe usable as a standalone library in other Django apps without requiring the host to copy dozens of settings.

The new module exposes a single scanpipe_settings object that resolves each value lazily in order: SCANPIPE dict in Django settings, then the SCANCODEIO_ environment variable, then the built-in default. Tests use override_settings(SCANPIPE={"KEY": value}) instead of flat SCANCODEIO_KEY overrides, which works automatically via Django's setting_changed signal.

scancodeio/settings.py now only contains infrastructure settings that Django needs at boot time: database, Redis, logging, REST framework configuration.

[JonoYang]

Some notes when using this branch of scio in purldb:

• I had to add taggit in addition to scanpipe in INSTALLED_APPS for a migration
• I had to add the following envvars to the purldb settings, as some pipelines import variables from the settings file:

PURLDB_URL = ""
PURLDB_USER = ""
PURLDB_PASSWORD = ""
PURLDB_API_KEY = ""
MATCHCODEIO_URL = ""
MATCHCODEIO_USER = ""
MATCHCODEIO_PASSWORD = ""
MATCHCODEIO_API_KEY = ""
VULNERABLECODE_URL = ""
VULNERABLECODE_USER = ""
VULNERABLECODE_PASSWORD = ""
VULNERABLECODE_API_KEY = ""
VULNERABLECODE_USER_AGENT = ""

• I am now failing some purldb api tests with the error message minecode/tests/test_api.py::ScannableURIAPITestCase::test_api_scannable_uri_get_next_download_url - django.db.utils.ProgrammingError: relation "authtoken_token" does not exist. I do not get this error when I use scancodeio from pypi

[tdruez]

I had to add taggit in addition to scanpipe in INSTALLED_APPS for a migration

Dependency on taggit removed in #2189
Taggit had also consequence on performances and not needed for our basic "label" usage.

I had to add the following envvars to the purldb settings, as some pipelines import variables from the settings file:

Ive updated the code in ac93c1d so you do not have to defined those in your main settings.

I am now failing some purldb api tests with the error message minecode/tests/test_api.py::ScannableURIAPITestCase::test_api_scannable_uri_get_next_download_url - django.db.utils.ProgrammingError: relation "authtoken_token" does not exist. I do not get this error when I use scancodeio from pypi

The dependency on authtoken_token was removed in #2087
See 79d7927#diff-f11933ef5e81f3fbaf94814cdbab8cfefa04a17d5ee619df9d659b8e71604450

I do not get this error when I use scancodeio from pypi

Could you confirm why exact version is running?