> ## Documentation Index > Fetch the complete documentation index at: https://docs.scanoss.com/llms.txt > Use this file to discover all available pages before exploring further. # Evaluation Walkthrough > A complete, hands-on walkthrough for evaluating ai-finder-pro: install it, run your first scan against your own project, save a machine-readable result, and learn how to read its confidence scores correctly. If your codebase uses AI — a locally-stored model file, a call to an LLM SDK, a LangChain agent, a vector database — most existing security and compliance tooling has no idea it's there. `ai-finder-pro` is a command-line scanner built to close that gap: point it at a repository, and it tells you which files are AI models (and, where possible, exactly *which* published model each one is), plus every place your source code touches AI — SDK imports, agent frameworks, tool definitions, vector stores, dataset references, and dependency manifests. This guide walks through the entire process end to end, assuming you've never touched the tool before: installing it, running your first scan, saving a complete machine-readable result, and — the part most guides skip — actually understanding what the numbers in that result mean and where they can mislead you if you don't read them carefully. Every command and every piece of output below is real, taken from an actual run. ## What You'll Learn By the end of this guide, you will be able to: * Install `ai-finder-pro` and confirm it's working correctly * Run a scan against your own project and read its terminal output * Point the tool at your own matching service, and avoid the one way this scan can silently give you results from the wrong place * Save a scan as a complete JSON file, and know what every major field in it means * Tell the difference between a result you can trust as fact and one that's an educated guess — a distinction that matters a great deal if you ever use this for compliance or audit purposes ## Workflow Overview Here's the full path from a fresh download to a result you can actually interpret. ```mermaid theme={null} graph TD Install[Install ai-finder-pro] Install --> Confirm[Confirm the Install] Confirm --> Scan[Run Your First Scan] Scan --> Save[Save Results as JSON] Save --> Understand[Understand Your Results] Understand --> Next[Next Steps] %% Links click Install "#step-1-install-ai-finder-pro" click Confirm "#step-2-confirm-the-install" click Scan "#step-3-run-your-first-scan" click Save "#step-4-save-your-results-as-json" click Understand "#understanding-your-results" click Next "#where-to-go-from-here" %% Styling style Install fill:#42A5F5,stroke:#1976D2,stroke-width:3px,color:#fff style Confirm fill:#42A5F5,stroke:#1565C0,stroke-width:2px,color:#fff style Scan fill:#66BB6A,stroke:#43A047,stroke-width:3px,color:#fff style Save fill:#26A69A,stroke:#00897B,stroke-width:2px,color:#fff style Understand fill:#FF9800,stroke:#F57C00,stroke-width:2px,color:#fff style Next fill:#66BB6A,stroke:#43A047,stroke-width:3px,color:#fff ``` Select any step in the diagram to jump straight to it. If you already have the tool installed and confirmed working, skip ahead to [Step 3: Run Your First Scan](#step-3-run-your-first-scan). ## Prerequisites * **Python 3.11 or newer.** * **Network access to PyPI**, the public Python package index. `ai-finder-pro` and its supporting packages — around forty in total, including `numpy` and an SBOM validation stack — install from PyPI in the ordinary way. No special credential or private package index is required. * **A C compiler toolchain**, needed to build one particular dependency (`py-tlsh`, a fuzzy-hashing library) from source if a prebuilt binary isn't available for your platform. On Debian/Ubuntu-derived Linux, install it with: ```bash theme={null} sudo apt install build-essential python3-dev ``` ## Step 1: Install ai-finder-pro Install `ai-finder-pro` from PyPI: ```bash theme={null} # Install from PyPI pip install ai-finder-pro # Upgrade an existing installation pip install --upgrade ai-finder-pro ``` This step can take a few minutes, since it's installing a real dependency tree, not just one package. ## Step 2: Confirm the Install ```bash theme={null} ai-finder-pro version ``` ``` ai-finder-pro 0.0.10 fingerprint_versions_supported = ['1.0', '2.0', '2.1', '3.0', '3.1', '3.2', '3.3', '3.5'] ai-fingerprint 3.17.0 ``` Your own version numbers will differ. Seeing all three lines confirms both packages installed and can find each other. It does **not** yet confirm that scanning actually works end to end — the [first scan](#step-3-run-your-first-scan) further down is what actually proves that. ### Configure the Matching Service Full model *identification* (not just detection of a file's presence, but confirming which published model it actually is) uses a matching service that the tool queries with a fingerprint of each model file it finds. Do this before you rely on any identification result — it's the step people skip, and skipping it fails silently. If you haven't pointed the tool at a service of your own, it falls back to a shared default (`https://aimatcher.scanoss.com`) automatically, with no error. That's fine if you're only trying the tool out. But if you *have* been given your own matching service, forgetting this step doesn't make your scans fail — it makes them keep working, quietly returning results from the wrong service instead of yours. Point the tool at your own service and key with: ```bash theme={null} ai-finder-pro config --apiurl ai-finder-pro config --key ``` These are written to a config file at `~/.config/ai-finder-pro/config.toml` (or under `$XDG_CONFIG_HOME`, if set). You can also set `SCANOSS_AI_SERVER_URL` and `SCANOSS_AI_API_KEY` as environment variables instead — usually the better choice in CI, since it keeps the key out of a file on disk. A flag passed on a single command wins over the environment variable, which wins over the config file, which wins over the built-in default. Check what's actually going to be used with: ```bash theme={null} ai-finder-pro config ``` ``` config_path = /home/you/.config/ai-finder-pro/config.toml server_url = https://aimatcher.scanoss.com api_key = cache_ttl_seconds = 86400 ``` The API key is masked when set. Running `config` with no flags only prints the resolved values — it doesn't change anything. **If you're only trying the tool out and haven't been given a service URL or key of your own, you don't need to do anything here.** A scan against the default fallback still works, still reports every source-code finding, and still tells you which files are model files. ### Set Up a Local Knowledge Base (Optional) A local knowledge base gives the scanner two things it doesn't otherwise have: a **pre-filter** that identifies well-named model files by name before it does the more expensive work of reading their weights, and an **offline fallback** for when the matching service is briefly unreachable. It isn't bundled with `ai-finder-pro`, and no default path is assumed. Build it with the separate OSS `ai-finder` CLI, installed as a dependency: ```bash theme={null} ai-finder kb init # create the database ai-finder kb update # populate it from the seed data server ai-finder kb status # confirm what it holds ``` Then tell `ai-finder-pro` where it is, either once in the environment or per scan: ```bash theme={null} export AI_FINDER_KB_PATH=~/.ai-finder/kb/ai-finder.db # or, per scan: ai-finder-pro scan /path/to/your/project --kb-path ~/.ai-finder/kb/ai-finder.db ``` This step is optional — everything else in this guide works without it — but it's worth doing now rather than later: the offline fallback is what saves a scan when the matching service is briefly unreachable, and a knowledge base you haven't built yet can't help you in that moment. ## Step 3: Run Your First Scan Point the scanner at your own project — no setup or sample data required: ```bash theme={null} ai-finder-pro scan /path/to/your/project --format table ``` Here's an example of what that output looks like, from a scan of an application whose model files are all called `model.safetensors`, in different directories: ``` PATH PURL CONF VIA ------------------------------------------------------------------------------------------------------------------------------------------- backend/app/agents/tools/summarizer/model.safetensors pkg:huggingface/distilbert/distilgpt2 1.00 sha256 backend/app/data/checkpoints/deepseek-r1/model.safetensors pkg:huggingface/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B 1.00 sha256 backend/app/services/math_solver/model.safetensors pkg:huggingface/Qwen/Qwen2.5-Math-1.5B 1.00 sha256 backend/scripts/experimental/deepseek-r1-tuned/model.safetensors pkg:huggingface/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B 0.50 composite model_files=4 hits=4 (api=4 kb=0) misses=0 source_files=373 Source findings: ---------------------------------------- agent (1): backend/app/agents/orchestrator.py:19 strands manifest_dep (9): requirements.txt:10 openai>=1.60.0 sdk_usage (16): backend/app/agents/model_provider.py:3 strands tool (13): backend/app/agents/orchestrator.py:790 @tool(context=True) ``` (One example line is shown per category above — the real output lists every finding.) Here's how to read the summary line: * **`model_files`** — model-file paths discovered, including any that failed fingerprinting. * **`hits`** — models identified, broken down by method in parentheses: `api` (queried against the matching service) and `kb` (matched against a local, offline knowledge base, if you've set one up). * **`misses`** — files that couldn't be identified, whether they were simply unrecognized or failed outright during processing. * **`source_files`** — files examined for source-code findings such as SDK usage, agent-framework code, or tool definitions; shown only when at least one was scanned. A scan that finds nothing still exits `0` — an empty result isn't an error. Check `misses` for what went unidentified. A couple of things worth understanding about output like this, since they reflect how the scanner behaves in general rather than anything specific to one run: * Three of the four rows matched at `1.00` via `sha256` — an exact byte match, the strongest evidence this tool can produce. The fourth matched at only `0.50` via `composite`: its bytes didn't match anything known, but several structural measurements agreed with `DeepSeek-R1-Distill-Qwen-1.5B`. Read a mid-confidence composite match as "derived from" or "a modified copy of," never as "is." The [Understanding Your Results](#understanding-your-results) section below goes into this distinction in depth, because it matters a great deal. * Not every scan comes back this clean. A real run can also produce misses — model files the scanner found but couldn't identify, because of an unsupported format, a corrupted header, or no usable layers. **A miss with a stated reason is a correct, working result — not a scan failure.** ## Step 4: Save Your Results as JSON The table view above is meant for reading at a glance in a terminal — it truncates long lists and shows only one example per category of finding. For anything you intend to keep, feed into another tool, or attach to a report, save the complete result as JSON instead: ```bash theme={null} ai-finder-pro scan /path/to/your/project --format json --output results.json ``` ``` wrote results.json ``` The `wrote ` confirmation is printed to stderr, which keeps stdout clean if you're piping the scan into another tool. If a scan does hit a fingerprint failure, the same diagnostic lines shown in the table view print above this regardless of output format — they describe what happened during the scan itself, not how you're viewing the result afterward. ### What's Inside the JSON File `results.json` is a single JSON object. The keys you'll use most often: | Key | What it holds | | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `scanned_paths` | The path (or paths) the scan was pointed at. | | `hits` | Models identified via the matching service or a local knowledge base. Normally content-derived — but can also include a `matched_via: "filename"` entry from an offline fallback, so this list isn't synonymous with "verified by content." | | `kb_hits` | Models identified purely by matching a filename against a local knowledge base, before the file's content is read. Always a guess based on the name, never a verification of content. | | `misses` | Files that couldn't be identified. Includes outright failures (`status: "error"`, with an `error` string explaining exactly why), and can also include a row where the service matched a fingerprint but simply had no candidate to offer (`status: "matched"`, no result) — count this list rather than filtering it by status. | | `source_findings` | Every source-code finding, in full — not the one-example-per-category sample the table view shows. | | `warnings` | Advisory messages about the scan itself — for example, a case where two candidate matches were equally likely and the tool declined to guess between them. | Here's an example hit, trimmed down to the fields worth knowing: ```json theme={null} { "path": "backend/app/agents/tools/summarizer/model.safetensors", "sha256": "e1ff18884359fe8beb795a5f414feb85a6ce3d929ad019c0d958c039d2b94a1b", "status": "matched", "confidence": 1.0, "matcher_version": "0.5.0", "purl": "pkg:huggingface/distilbert/distilgpt2", "matched_via": "sha256", "matches": [ { "purl": "pkg:huggingface/distilbert/distilgpt2", "name": "distilgpt2", "matched_via": "sha256", "confidence": 1.0, "confidence_band": "high", "license": "Apache-2.0", "architecture": "GPT2LMHeadModel", "parameter_count": 88204032, "datasets": ["openwebtext"], "base_model_purl": null, "provenance": { "authoritative_host": "pkg:huggingface/distilbert/distilgpt2", "also_hosted_at": [], "canonical_status": "asserted", "host_authority_ambiguity": false } } ] } ``` A few fields worth flagging, beyond the ones already covered above: * **`matches` is ordered best-first, and `matches[0]` is the candidate the tool chose** — the top-level `purl` and `matched_via` are copied from it. If you're checking a policy against a specific candidate, `matches[0]` is the one that matters; anything after it is a candidate the tool considered and didn't pick. * **`base_model_purl`** names the parent a fine-tuned model was derived from, when the reference record knows it — `null` here, because `distilgpt2` is a base model rather than a fine-tune. * **`provenance`** tells you where a model is actually published, which matters if your copy came from a mirror. `host_authority_ambiguity: true` means the tool couldn't settle the origin between two competing hosts — **that's the field to alert on** if provenance matters to your use case. `hits` and `kb_hits` never overlap — a given result appears in exactly one of them. But `hits` isn't purely content-verified either: alongside matching-service results, it can also hold a `matched_via: "filename"` entry from an offline fallback. If your own tooling needs to tell "identified by content" apart from "identified by name," check `matched_via` (or the `identification` field described below) on each result — don't assume everything sitting in `hits` cleared a higher bar than everything in `kb_hits`. ## Understanding Your Results This is the part that matters most if you plan to use this tool's output for anything beyond curiosity — a compliance filing, a security gate, an inventory you'll actually rely on. A few things are very easy to misread on a first pass. **Confidence is a category, not a percentage.** A score of `1.0` means the scanned file's bytes are byte-for-byte identical to a known, published model — the strongest evidence this tool can produce. A score of `0.5` does *not* mean "50% similar." It means the file's internal structure statistically resembles a known model across several independent measurements, without the bytes matching. Don't build a policy threshold that treats confidence as a sliding distance metric — a `0.5` composite match and a `0.9` you might imagine from some other tool are not comparable in that way, because this scanner doesn't inflate structural similarity into a false sense of exactness. **`matched_via` tells you *how* an identification was reached, and that matters as much as the confidence number:** | `matched_via` | What it means | | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `sha256` | The file's exact bytes matched a known model. As strong as identification gets — this is the only basis for asserting "this file *is* that model." | | `composite` | The bytes didn't match anything known, but several structural measurements agreed with a known model. Read this as "derived from" or "a modified copy of," never as "is." | | `filename` | The identification came entirely from the file's name, resolved against a local database, before the tool ever read the file's actual contents. Confidence here is deliberately capped low (typically `0.35`, or `0.30` from an offline fallback) because a name is not evidence about what's inside a file — and these lookups can be flatly wrong, matching on a substring of an unrelated model's name. | The full set of `matched_via` values is `sha256`, `md5`, `header_sha256`, `manifest_sha256`, `tlsh`, `composite`, and `filename`. Every one of those except `filename` is derived from the file's actual content; only `filename` is a name-based guess. That's exactly what the **`identification`** field also tells you, in a single value, if you'd rather check one field than reason about seven possible `matched_via` values: `content-fingerprint` for any hash-based or `composite` match, `name-heuristic` for a `filename` match, and `unresolved` when nothing matched at all. Don't read `content-fingerprint` as a synonym for "exact copy," though — it's a broad category that includes structural matches like `composite`, not just `sha256`. **A miss with a stated reason is a correct result, not a bug.** Most entries in `misses` carry an `error` string that says exactly why identification didn't happen — an unsupported file format, no usable layers, a corrupted header. Less obviously, a miss can also show `status: "matched"` with no result at all: the matching service processed the fingerprint successfully but simply had no candidate to offer. Because of that, count the `misses` list rather than filtering it by `status`. **A `kb` hit is a lead, not a verified fact.** It comes from matching a filename against a local database, before any of the file's actual content is examined. Treat it as a cheap, useful first pass for triage — not as evidence you'd cite in an audit. The practical rule to take away: if a decision depends on asserting *which model you actually have* — for a compliance record, a licensing decision, anything where being wrong has consequences — require an exact match (`matched_via: sha256`, confidence `1.0`). Anything else, however confident-sounding the number looks, is a lead worth investigating further, not a finding you can rely on as-is. ## Where to Go From Here At this point you've installed the tool, run a scan against your own project, saved a complete machine-readable result, and — the part that actually matters — learned how to read that result critically instead of taking every number at face value. From here, a few natural next steps: * **Try other output formats.** Beyond `table` and `json`, the tool can emit industry-standard bill-of-materials formats — CycloneDX (`--format cyclonedx`) and SPDX 3.0.1 (`--format spdx3`) — for feeding into existing compliance or SBOM tooling. Each format has its own tradeoffs worth understanding before you rely on one (in particular, both bill-of-materials formats deduplicate identical models, which can hide a modified copy sitting alongside a genuine one — the JSON format shown in this guide is the one to use if you need a complete, per-file record). * **If you skipped the local knowledge base earlier, set it up now** — see [Set Up a Local Knowledge Base](#set-up-a-local-knowledge-base-optional) above. It's a quick win once you're past evaluating the tool: a filename pre-filter that's cheaper than fingerprinting every file, plus an offline fallback if the matching service is briefly unreachable. * **Wire it into CI**, gating a build on whether any model failed to identify cleanly, or on whether a scan quietly fell back to a weaker identification method than you expected. If you found this useful, the single habit worth keeping from all of this: read `matched_via` before you trust `confidence`. That one distinction is where most misunderstandings about what this tool is telling you actually come from. Need help? [Contact our AI assistant](?assistant=open)