> For the complete documentation index, see [llms.txt](https://docs.ipcheck.ing/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.ipcheck.ing/developer/architecture/ip-data-sources.md).

# IP Data Sources

Two questions drive most of MyIP, and they are answered by different machinery:

* **"What is my IP?"** — the browser asks several third-party echo endpoints directly. The backend is not involved.
* **"Where is this IP?"** — the browser asks *our* backend, which queries one geolocation provider (or a local database) and normalizes the answer.

Keeping them separate is deliberate: an echo endpoint must see the visitor's own connection, so it cannot be proxied.

```mermaid
flowchart TD
    subgraph Browser
      G["utils/getips/ — one function per source"]
      T["utils/transform-ip-data.js"]
      C["IP cards"]
    end
    subgraph Backend
      H["Geolocation handlers in api/<br/>ipinfo-io, ipapi-com, ipapi-is,<br/>ip2location-io, ip-sb, ipcheck-ing, maxmind"]
      M["common/maxmind-service.js — local mmdb"]
    end
    P["Third-party echo endpoints<br/>Cloudflare, IPCheck.ing, IPIP.net, ..."]
    U["Geolocation providers"]

    G -->|"direct fetch"| P
    G -->|"the IP"| H
    H --> U
    H --> M
    H -->|"canonical JSON"| T --> C
```

## Step 1 — resolving your own IP

`frontend/utils/getips/` holds one small module per source. Each exports an `async` function returning `{ ip, source }`, validates the result with `isValidIP()` from `common/valid-ip.js`, and goes through `fetchWithTimeout` (5-second browser default).

`frontend/components/IpInfos.vue` renders up to six cards and assigns one source function to each, by index:

| Card | Primary source     | Endpoint                               | Falls back to                                   |
| ---- | ------------------ | -------------------------------------- | ----------------------------------------------- |
| 0    | IPCheck.ing IPv4   | `4.ipcheck.ing`                        | IPify IPv4 (`api4.ipify.org`)                   |
| 1    | IPCheck.ing IPv6   | `6.ipcheck.ing`                        | IPify IPv6 (`api6.ipify.org`)                   |
| 2    | Cloudflare IPv4    | `1.0.0.1/cdn-cgi/trace`                | MyExternalIP IPv4                               |
| 3    | Cloudflare IPv6    | `[2606:4700:4700::1111]/cdn-cgi/trace` | MyExternalIP IPv6                               |
| 4    | IPIP.net           | `myip.ipip.net/json`                   | Upai (`pubstatic.b0.upaiyun.com`)               |
| 5    | IPCheck.ing IPv6/4 | `64.ipcheck.ing`                       | — (JSON, then the same host's `/cdn-cgi/trace`) |

{% hint style="info" %}
The three IPCheck.ing sources take an `originalSite` argument. On the canonical deployment they read the JSON endpoint; elsewhere they read the Cloudflare-style `/cdn-cgi/trace` text of the same host. If the JSON call fails they retry via trace before giving up.
{% endhint %}

Failure handling is layered:

1. A source that throws or returns an invalid IP logs a `console.warn` and returns its declared fallback, or `{ ip: null, source }`.
2. Every card runs its own resolve-then-detail pipeline, and all six run under `Promise.allSettled`, so a dead source cannot sink the batch. Cards paint independently as they land.
3. When a card's whole chain fails, the SPA emits an `ip-source:exhausted` event on the app bus. Sentry (when configured) captures it only if another card of the same IP version did resolve — otherwise "our chain failed" is indistinguishable from a visitor with no IPv6, which is routine noise.

Individual source failures are `console.warn` by design, so they never reach error monitoring; the per-card exhaustion event is the health signal.

## Step 2 — geolocating an IP

Once a card has an IP it calls the backend. `frontend/data/ip-databases.js` is the registry of selectable sources:

| id | Name           | Endpoint                                  | Needs a key                            |
| -- | -------------- | ----------------------------------------- | -------------------------------------- |
| 0  | IPCheck.ing    | `/api/ipchecking?ip={{ip}}&lang={{lang}}` | `IPCHECKING_API_KEY` (private API)     |
| 1  | IPinfo.io      | `/api/ipinfo?ip={{ip}}`                   | Optional (`IPINFO_API_KEY`)            |
| 2  | IP-API.com     | `/api/ipapicom?ip={{ip}}&lang={{lang}}`   | No                                     |
| 3  | IPAPI.is       | `/api/ipapiis?ip={{ip}}`                  | `IPAPIIS_API_KEY`                      |
| 4  | IP2Location.io | `/api/ip2location?ip={{ip}}`              | `IP2LOCATION_API_KEY`                  |
| 5  | IP.sb          | `/api/ipsb?ip={{ip}}`                     | No                                     |
| 6  | MaxMind        | `/api/maxmind?ip={{ip}}&lang={{lang}}`    | Local database, no key at request time |

`buildDbUrl(db, ip, lang)` substitutes the `{{ip}}` and `{{lang}}` placeholders. Each source's `enabled` flag is derived from the `/api/configs` feature flags when they load (`applyConfigAvailability`) — key-gated sources follow their flag, key-free ones are always available — and runtime fetch failures never touch it. Users switch source in Preferences; if a stored choice points at a source that is no longer configured, it is migrated to the nearest available one (`nearestEnabledId`, walking forward in id order) with a toast — the only case where the stored preference is rewritten. Configuring keys is covered in [Optional API Keys](/developer/configuration/optional-api-keys.md).

### Client-side fallback chain

`IpInfos.vue` does not simply query the preferred source and give up. Inside `fetchIPDetails()`:

* The requested source is looked up in the **enabled** list; if it is absent (its config flag is off, or configs have not loaded yet), the walk starts at the first enabled one.
* On an error it logs, advances to the next enabled source, and retries — until every enabled source has been attempted.
* Landing on a different source than requested shows a one-time toast and drifts the session's runtime source, so later queries start from the one that works — the stored preference is never rewritten.
* Results are cached per IP, and in-flight requests are deduplicated by IP, so six cards showing the same address produce one request.

## The canonical response shape

Every geolocation handler returns the same JSON, whatever the upstream looked like. From `api/ipinfo-io.js`:

```js
{
    ip,
    city,
    region,
    country,        // ISO 3166-1 alpha-2
    country_name,
    country_code,   // same code as `country`
    latitude,
    longitude,
    asn,            // "AS13335" — with the AS prefix
    org
}
```

| Field                      | Notes                                                                                     |
| -------------------------- | ----------------------------------------------------------------------------------------- |
| `ip`                       | Echoed back by the upstream                                                               |
| `city` / `region`          | Free-form strings; `'N/A'` when the upstream has nothing                                  |
| `country` / `country_code` | Both carry the ISO alpha-2 code                                                           |
| `country_name`             | Upstream's own name — the frontend usually replaces it                                    |
| `latitude` / `longitude`   | Numbers                                                                                   |
| `asn`                      | Normalized to the `AS<number>` form; sources returning a bare number get the prefix added |
| `org`                      | Organization or ISP name                                                                  |

Two sources extend it. `api/ipapi-is.js` adds `isHosting` and `isProxy` booleans. `api/ipcheck-ing.js` is a pass-through proxy to the private IPCheck.ing API and returns that API's payload verbatim, including an `advancedData` object the frontend unpacks into proxy, IP-type, native-IP, quality-score, protocol and provider fields.

<details>

<summary>What the frontend does with the payload</summary>

`frontend/utils/transform-ip-data.js` turns a canonical response into card data:

* `country_name` is re-derived from the country **code** locally so every source shows the same UI-language name; the upstream string is only a fallback.
* `country_code` of `'N/A'` becomes an empty string.
* `org` becomes `isp`, and an `asn` starting with `AS` gets an `asnlink` to bgp.tools.
* Coordinates are rounded to one decimal for `mapUrl` / `mapUrl_dark`, which point at `/api/map`. At the map's zoom level \~0.176° is one pixel, so 0.1° is sub-pixel — the marker looks identical while every IP in the same grid cell collapses onto one edge-cache key. The full-precision coordinates are kept for display.
* For source `0` (IPCheck.ing) it also extracts the `advancedData` fields.

</details>

Adding a source means writing a handler that produces this shape and adding a row to `data/ip-databases.js`. New handlers should use the `makeGeoHandler({ name, buildUrl, normalize })` factory in `common/geo-handler.js`, which owns the shared shell: read the already-validated `?ip`, fetch through `fetchUpstream`, throw on a non-2xx status (outage pages come back as HTML and would otherwise blow up `JSON.parse`), normalize, respond, and log-and-500 on failure.

## Local datasets

Two datasets live on disk inside `common/` and are read synchronously at request time. Both have an auto-updater that runs in the same process.

### MaxMind GeoLite2

`common/maxmind-service.js` opens `GeoLite2-City.mmdb` and `GeoLite2-ASN.mmdb` from `common/maxmind-db/` and keeps both readers in memory. `lookupMaxMind(ip, lang)` merges a City and an ASN record into the canonical shape, resolving localized names with an English then `'N/A'` fallback. If either reader is missing it throws with `statusCode` 503, and `/api/maxmind` answers 503 — the API degrades, the server still runs.

Updating is handled by `common/maxmind-updater.js`:

| Behavior               | Value                                                                                                  |
| ---------------------- | ------------------------------------------------------------------------------------------------------ |
| Bootstrap on boot      | Downloads only if the files are missing, capped at 5 minutes; runs regardless of `MAXMIND_AUTO_UPDATE` |
| First scheduled check  | 60 seconds after start                                                                                 |
| Repeat interval        | Every 24 hours                                                                                         |
| Gate for the scheduler | `MAXMIND_AUTO_UPDATE=true` plus `MAXMIND_ACCOUNT_ID` and `MAXMIND_LICENSE_KEY`                         |
| Concurrency            | A lock file, considered stale after 2 hours                                                            |

Downloads are staged and published atomically. A separate file watcher (`startMaxMindFileWatcher()`) polls both files every 5 seconds and reloads the readers when another process replaces them, debounced by 1 second so City and ASN land as a single reload. If the new file is invalid the existing readers stay in place. Setup instructions live in [MaxMind Setup](/developer/getting-started/maxmind-setup.md).

### CAIDA as2org and AS relationships

`common/caida-updater.js` manages two datasets with the same lock / state / atomic-publish / validate / reload machinery:

| Dataset  | File                               | Source                                                                                   | Used by                        |
| -------- | ---------------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------ |
| `as2org` | `common/as-org-db/as-org2info.txt` | `publicdata.caida.org/datasets/as-organizations/latest.as-org2info.txt.gz`               | AS → organization name lookups |
| `as-rel` | `common/as-rel-db/as-rel2.txt`     | Newest `*.as-rel2.txt.bz2` in `publicdata.caida.org/datasets/as-relationships/serial-2/` | The AS connectivity graph      |

`as2org` has a stable `latest` symlink upstream, so a `HEAD` plus `Last-Modified` is enough to detect a new snapshot. `as-rel` has none, so the updater scrapes the directory listing and takes the lexicographically newest `YYYYMMDD` filename. Both are decompressed on the fly and validated before being published.

Schedule mirrors MaxMind: bootstrap on boot if missing (2-minute cap), first check 60 seconds after start, then every 24 hours, with the periodic scheduler gated on `CAIDA_AUTO_UPDATE=true`. Bootstrap always runs so a fresh checkout works.

`common/as-rel-db.js` parses the pipe-delimited rows and keeps only the provider-to-customer (`-1`) relationships, building a customer → providers index and a customer count per AS. The Tier 1 set is derived from the snapshot rather than hardcoded: an AS with no providers in the p2c topology that also provides transit to at least 100 others. `/api/asn-connectivity` then does a fully local, synchronous BFS from the origin AS out to Tier 1s.

`common/as-org-db.js` parses CAIDA's pipe-delimited TXT (\~12 MB) instead of the equivalent JSONL (\~28 MB) — identical content, and `split('|')` beats `JSON.parse` per line by roughly 40%. Both modules pick the newest matching file by modification time, so a manually downloaded snapshot with a different name still works.

## What happens when something fails

| Failure                                  | Result                                                                                                           |
| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| One echo endpoint is down                | Its module falls back to a second endpoint; if that fails the card shows nothing and emits `ip-source:exhausted` |
| One geolocation provider errors          | The client falls back to the next enabled source and shows a toast; later queries start from the working source  |
| Every geolocation source fails for an IP | The card's detail fields stay empty; the error is logged client-side                                             |
| An upstream hangs                        | `fetchUpstream` aborts at 8 seconds and the handler returns `500 { error }`                                      |
| MaxMind databases missing or invalid     | `/api/maxmind` returns 503; the rest of the API is unaffected                                                    |
| CAIDA snapshots missing                  | The connectivity graph comes back empty, and org-name lookups fall back to RIPEstat's `as-overview`              |
| An upstream returns an HTML outage page  | `makeGeoHandler` throws on the non-2xx status before parsing                                                     |

## Related pages

* [Backend](/developer/architecture/backend.md) — guards, caching tiers, and the boot sequence
* [Optional API Keys](/developer/configuration/optional-api-keys.md) — which sources need credentials
* [MaxMind Setup](/developer/getting-started/maxmind-setup.md) — obtaining and refreshing GeoLite2
* [API Endpoints](/developer/reference/api-endpoints.md) — the full request/response contract


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.ipcheck.ing/developer/architecture/ip-data-sources.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.