Sources, keys and building your own

What a source is

A source — a connector — is one thing this platform knows how to ask. It declares which entity types it accepts, and the fan-out runs it against every entity of those types that it meets.

There are around a hundred. Browse them: each says what it does, what it accepts, and whether it is available to you right now.

Why a source is greyed out

Almost always a missing API key, and that is a state worth distinguishing from "found nothing".

A source that quietly did not run and a source that ran and found nothing look identical in a result. Only one of them is worth doing something about. So a keyless source is shown as unavailable, the Sources tab in the inspector says which sources were skipped and why, and the "scan with source" submenu lists key-gated sources disabled with a key pill rather than hiding them.

A missing key never breaks an investigation. It disables one source and the scan runs without it.

Your API keys

Keys live encrypted on your account, and a scan spends the keys of whoever owns the investigation — not whoever happened to open the page.

This used to be one shared set for the whole platform. It is not, because Shodan and HIBP bill per key and a shared key spends one researcher's quota on another's investigation.

Add them under Account & API keys. They take effect on the next request — no restart, no waiting. A source you just added a key for lights up immediately, and Re-run every selected source on a node is how you get the answer you were missing without starting again.

Cached answers and your quota

The platform caches a source's conclusion about an entity, so a second investigation of the same subject is dramatically cheaper. Two rules keep that honest:

• Entitlement. A key-gated source's cached answer is only served to somebody who holds that key. Otherwise one researcher's paid quota would subsidise everybody else's work.
• Disclosure. A replayed source is flagged as cached on the task and shown as such in the activity ticker. A report must never imply a source was queried when it was not.

Nothing is kept longer than a day, whatever the source says.

Source groups

Covered in detail in the research guide, but the short version: an investigation carries a selection chosen before the first task is queued, it is enforced on every ring rather than just the seed, and the report says which group ran and adds a limitation line when the selection was narrowed.

Key-gated sources stay in a selection whether or not you hold the key — whether a key is present is answered later, per request, by the source itself. Filtering them out at the start would bake one researcher's keyring into another's investigation.

Sources that need no key at all

Some sources cost CPU here rather than money anywhere. No third party is told what you are investigating, there is no quota, and there is no key. If one of those reports itself as temporarily unavailable, that is an operator problem rather than anything you can fix from your account.

How to read a source that did not answer

This is the recurring lesson of the whole platform and it is worth stating in one place, because two real bugs came out of getting it wrong:

• A refusal inside a success is still a refusal. One Danish company register answers HTTP 200 with an error body when you are out of quota. Reading that as data reported every Danish company as unregistered — and because the answer looked successful, it was cached for a week.
• A dead source answers "nothing found" forever. One blocklist zone was shut down in 2024 and still resolves, returning "not listed" for every address — it would have voted clean on everything in perpetuity. Blocklist sources now check each zone's own test entry and report a zone that will not list its own canary as unavailable rather than as agreement.

When you read a result, the question is not "did it find anything" but "was it in a position to find anything". The Sources tab answers that.

Building your own source

Security Specialist and above. Point the platform at an API of your own through a web form; it is stored in the database, scoped to you or your organisation, and merged into the catalogue for your scans only.

How the builder works

1. Name it, say which entity types it accepts, and give it a URL with {input} where the entity goes.
2. Add any headers it needs. They are stored encrypted and come back masked.
3. Press Test. It makes one real call, and the value picker is built from the response that actually came back rather than one you imagined.
4. Pick values off the response to map them onto entities. Picking a value writes the path — it is a picked path, not an expression you have to get right in the dark.

The mapping tool distinguishes "the list was empty" from "your mapping is wrong" from "the document was empty". Collapsing those three is the same bug as reading an error body as data, one layer down.

What it will refuse

The URL is free text by decision — the point is that nobody has to approve your vendor first. That makes the outbound request guard the whole defence, so the builder enforces everything that guard does not:

• HTTPS only, GET or POST only.
• No Host, Content-Length, Transfer-Encoding or Cookie headers; no line breaks in a header value.
• {input} is percent-encoded before it goes anywhere near the URL.
• Redirects are not followed.

Remember what you are doing: the requests leave from this server's address. An endpoint that objects to being polled objects to this platform, not to you. Administrators can see every custom source on the instance for exactly that reason.