# Sources

> The sources list and a source's page, descriptions, environments, source tokens, arrival lag, merging and deleting.

## What a source is

A source is one stream of logs: usually a server, named after its hostname, or an application, a script or an AWS forwarder that sends over HTTPS. Every event belongs to one source.

A source comes to exist in one of two ways:

- **A syslog server** becomes a source by itself the first time one of its lines reaches the account's port, named after the hostname in the message. There is nothing to create beforehand. A server sending from an address that is not trusted waits for approval, unless the account approves new syslog sources automatically - see [Approving senders](https://app.sanelog.com/docs/sending-logs.md#approving-senders).
- **An application, a script or an AWS forwarder** sends with a source token. Its source is created first, with **Add source** (see [Adding a source](https://app.sanelog.com/docs/sources.md#adding-a-source)).

How many sources the account can hold depends on the plan - see [Sources included](https://app.sanelog.com/docs/billing.md#sources-included).

### The sources list

Open **Sources > All sources** in the sidebar to see every source, with its **Environment**, **Last event**, **Events (24h)**, **Arrival lag**, **Token** status and when it was **Created**. A source waiting for approval is marked **Pending**.

The filters above the list narrow it by **Filter** (the sources a saved filter covers), **Environment**, **Approval**, **Description** (sources with or without one) and **Config set**.

## Adding a source

Choose **Add source** on the **All sources** page to create a source for an application, a script or an AWS forwarder. Syslog servers need no source created here.

The form asks for:

- **Name** - unique within the account, such as `checkout-app` or `payments-aws-logs`.
- **Environment** - optional. See [Environments](https://app.sanelog.com/docs/sources.md#environments).
- **Config set** - the settings for receiving the source's logs. See [Config sets](https://app.sanelog.com/docs/sending-logs.md#config-sets).
- **Description** - optional, but worth writing. See [Descriptions](https://app.sanelog.com/docs/sources.md#descriptions).

Saving issues the source's token, and the source's page shows it with a ready-to-run example. A source added this way is approved from the start, because its token identifies it.

Users with Read Only access can see sources but cannot add, edit, merge or delete them.

## A source's page

Choose a source in the list to open its page. At the top are **View in Log Explorer**, **Edit**, **Merge** and **Delete**, and a badge when the source is **Pending approval** or **Paused**.

The page shows:

- **Details** - the source's name, environment, the [filters](https://app.sanelog.com/docs/filters.md#how-filters-work) that cover it, whether it is receiving logs, its approval, its last event, its config set and its description. A source that has sent unencrypted syslog also shows **Last unencrypted**.
- **Logs received**, with charts of the events and bytes received every 5 minutes over the last 24 hours.
- **Arrival lag** - see [Arrival lag](https://app.sanelog.com/docs/sources.md#arrival-lag).
- **Senders** - for a syslog source, every address it has sent from, with the location the address is registered to, its status and when it was first and last seen.
- **Source token** - see [Source tokens](https://app.sanelog.com/docs/sources.md#source-tokens).
- **Activity** - when the source was created and last updated.
- **Dropped events (last 24 hours)** - events the source's [config set](https://app.sanelog.com/docs/sending-logs.md#config-sets) dropped, by reason.
- **Redaction matches (this month)** - see [Redaction](https://app.sanelog.com/docs/redaction.md#counting).

Callouts above the cards appear when there is something to act on: a sender waiting for approval, a source that looks like a duplicate of another, or unencrypted syslog.

## Editing a source

Open the source and choose **Edit** to change its name, environment, config set or description.

### Descriptions

A source's description says what the source is and what it is for. Teammates see it on the source's page, and [Explain line](https://app.sanelog.com/docs/log-explorer.md#explain-line) and coding agents read it to work out what each source is and how the sources relate.

A description that names the source's role and what sits around it gives better explanations:

- Poor: "A web server"
- Better: "One of three production Linux web servers behind an AWS load balancer and WAF"

### Pausing a source

**Pause receiving logs from this source** on the edit page refuses new logs sent to the source. Stored logs stay in the Log Explorer. An HTTP sender receives a `403` while the source is paused.

A paused source still counts toward the plan's sources. Switch the setting off to receive logs again.

## Environments

An environment is a tag on a source naming the deployment tier it belongs to, such as production, staging or development. You can use any text. The tag is stored in lower case, so `Prod` and `prod` are the same environment.

### Tagging sources

Set the tag with **Environment** on a source's form. The picker offers production, staging, development and every value already in use in the account. To add a new value, type it in the search box and choose the **Create** row that appears below the options.

A new source gets a suggested environment from its name, whether you add it yourself or it appears when its first syslog line arrives. The name is split into words at any character that is not a letter, and the first word that names an environment decides:

| Environment | Words |
|---|---|
| production | prod, production, prd, live |
| staging | stag, staging, stg, uat, preprod |
| development | dev, development, test, testing, local |

So `web-1.production` and `prod2` both suggest production, and `web-1` suggests nothing. On the form you can change the suggestion before saving. A source's environment is never guessed again after it is created.

A source with no environment is unassigned. Unassigned sources work normally but are not included by a filter's **Environments** field.

### Where environments work

- **Filters** can include sources by environment. A filter on production covers every production source, including sources tagged later. See [Scope: which sources](https://app.sanelog.com/docs/filters.md#scope-which-sources).
- **The Log Explorer's** source picker lists every environment in use, so you can view one environment without saving a filter.
- **The sources list** can be narrowed to one environment, or to **Unassigned**.
- **The API** returns each source's environment and accepts `environment` as a parameter, so an agent can tell staging events from production events without parsing source names.

### Renaming an environment

Environments have no rename step. An environment exists as long as at least one source carries it.

To rename one, change the tag on each source that uses it. The old name leaves the pickers once no source carries it, unless it is one of the three built-in values.

Filters that include the old name keep it until you edit them. Until then they no longer match the renamed sources.

## Source tokens

A source token lets an application, a script or an AWS forwarder send events to one source over HTTPS. The **Source token** card on the source's page shows the token whenever you need it, along with a ready-to-run `curl` example.

A source created by syslog has no token until you choose **Issue new token**.

- **Rotate token** issues a new token. The old token keeps working for 60 minutes so you can switch senders over one at a time, and the card shows when it stops as **Old token valid until**.
- **Revoke token** stops the token working at once.

A source token can only send logs. Reading the account's logs through the API takes an [API token](https://app.sanelog.com/docs/account.md#api-tokens).

## Arrival lag

**Arrival lag** is how long a line takes to reach SaneLog after the time the sender wrote on it. The source's page shows the **Typical line** and the **Slowest tenth**, and the sources list shows the same figure for every source.

The figure is measured over every line from the last 15 minutes, or over the source's newest 200 lines when those are more, within the last 24 hours. A source that sends once an hour therefore keeps a figure between sends.

A second or two is usual, because senders queue and batch their logs.

When the typical line is 10 seconds or more behind, over at least 20 lines, the dashboard lists the source under **Needs attention** for as long as the lag lasts. The sender is queuing, sending large batches, or running a slow clock. A sender whose clock runs fast shows as ahead, and is listed the same way.

A source with a large arrival lag also places its lines higher up the Log Explorer's live stream - see [Live tail order](https://app.sanelog.com/docs/log-explorer.md#live-tail-order).

## Merging duplicates

Two servers that report the same hostname become one source. To separate them, fix the hostname on one of the senders (or turn on `preserveFQDN` in rsyslog).

The opposite happens when a server's hostname changes, for example from `web-1` to `web-1.example.com`: the server appears as a new source and its history is split between two sources. To combine them, open the duplicate and choose **Merge**, pick the **Destination source** and type the duplicate's name to confirm.

- The duplicate's stored logs and log volume history move into the destination, and the duplicate is deleted.
- The move runs in the background and can take a while for a large source.
- Merging cannot be undone, and the moved events cannot be told apart from the destination's own events.
- If the duplicate is still receiving logs, receiving is paused when the merge starts, and anything sent to it during the merge is dropped.

When a source looks like a duplicate of another, based on its sender address and hostname, its page suggests the merge.

## Deleting a source

To stop a source accepting logs, open it and choose **Delete**, then type the source's name to confirm:

- Its stored logs stay in the Log Explorer until the retention period removes them.
- With **Delete any logs stored from this source** ticked, the stored logs are removed within a few minutes and cannot be recovered.
- If the server keeps sending syslog, its hostname appears again as a new source.

Deleting a source frees its place in the plan at once.