Send a complete list

Some systems can tell you when a record is deleted. Many can’t. A snapshot is for those: you send a complete list of the records the source holds, and Samesake deletes the ones that aren’t on it.

If your system can report deletions, send each one as it happens. A snapshot is then only a safety net, run now and then to catch anything missed.

How a snapshot works

There are three steps, and every call uses the source’s API key:

Open the snapshot. You get a snapshotId.
Send every record the source holds, in requests of up to 1,000, as many as you need.
Complete the snapshot. Samesake deletes the source’s records that weren’t in it.

A record is only deleted when you complete the snapshot. If you open one, send some records and stop, nothing is deleted.

Open a snapshot

curl -i -X POST https://app.samesake.co.uk/api/v1/snapshots \
  -H "X-Api-Key: mk_abcd1234_0123456789abcdefghjkmnpqrstvwxyz"

Try it in the API reference

You get 201 Created:

{ "snapshotId": "3f2b8c1e-9d4a-4e7b-a1c5-6d8e2f4b9a07" }

A source has one current snapshot. Opening a new one discards any earlier snapshot of the source that’s still open or held, so a stale run can never delete records that a newer list contains.

Send the records

Send the records to the snapshot, in a JSON array of up to 1,000, exactly as you would to a batch:

curl -i -X POST https://app.samesake.co.uk/api/v1/snapshots/3f2b8c1e-9d4a-4e7b-a1c5-6d8e2f4b9a07/records \
  -H "X-Api-Key: mk_abcd1234_0123456789abcdefghjkmnpqrstvwxyz" \
  -H "Content-Type: application/json" \
  -d '[
    {
      "sourceRecordId": "CRM-10042",
      "firstName": "Priya",
      "lastName": "Sharma",
      "email": "priya.sharma@example.com",
      "phones": ["07700 900123"]
    },
    {
      "sourceRecordId": "CRM-10043",
      "title": "Mr",
      "firstName": "Tom",
      "lastName": "Fletcher",
      "dateOfBirth": "12/09/1979",
      "postcode": "M1 4BT",
      "phones": ["07700 900456"]
    }
  ]'

Try it in the API reference

Each record is created or updated straight away, and counts as being in the list. The response is the same as a batch’s: one result for each record, in order. The limits are a batch’s too: 1,000 records and 8 MB for a request, and a request with an invalid record is refused whole.

Send the whole list, across as many requests as it takes. Each record once is enough, and sending one again, on a retry for instance, is harmless: the snapshot simply sees it again.

Complete the snapshot

curl -i -X POST https://app.samesake.co.uk/api/v1/snapshots/3f2b8c1e-9d4a-4e7b-a1c5-6d8e2f4b9a07/complete \
  -H "X-Api-Key: mk_abcd1234_0123456789abcdefghjkmnpqrstvwxyz"

Try it in the API reference

{ "status": "Applied", "missingCount": 3 }

missingCount is how many of the source’s records weren’t in the snapshot. status says what happened to them:

Status	Meaning
`Applied`	The missing records were deleted.
`Held`	Too many were missing, so nothing was deleted. An admin decides.

Complete a snapshot once. Completing it again gets 409 Conflict, because it’s no longer open.

The 10% hold

A snapshot that goes wrong, such as an export that stops half way, could look like a system that deleted most of its customers. So Samesake holds back: if completing the snapshot would delete more than 10% of the source’s live records, it deletes nothing, and the status is Held.

The hold is proportionate, so it bites hardest on small sources: with fewer than 10 live records, any deletion at all is held, because one record is already more than 10%.

An admin then decides, on Admin, then Matching, under Held snapshots:

Delete the records, if the list really was complete. Samesake works out which records are missing at that moment, and deletes them.
Discard snapshot, if it wasn’t. Nothing is deleted. The records you sent are kept as they are.

Opening a newer snapshot for the source also discards a held one.

If you expect a big, genuine drop, such as a clear-out of old customers, send the deletes with DELETE for the ones you know about, or ask an admin to approve the held snapshot.

Changes made while it runs

A snapshot never deletes a record that was created or changed after the snapshot was opened, by any other call. So your system can keep sending changes while a long snapshot runs: a customer who joined after you opened it can’t be deleted by it.

A complete script

This Python script sends a list and completes the snapshot. It retries what’s safe to retry, as Errors and limits describes. It needs the requests library (pip install requests).

import time

import requests

BASE = "https://app.samesake.co.uk/api/v1"
HEADERS = {"X-Api-Key": "mk_abcd1234_0123456789abcdefghjkmnpqrstvwxyz"}


def send(method, path, **kwargs):
    """Send one request. Retries a 429 (after Retry-After), a 5xx, a network error, and a 409 on a record call."""
    for attempt in range(6):
        try:
            response = requests.request(method, BASE + path, headers=HEADERS, timeout=90, **kwargs)
        except (requests.ConnectionError, requests.Timeout):
            time.sleep(2 ** attempt)
            continue
        status = response.status_code
        if status == 429:
            time.sleep(int(response.headers.get("Retry-After", 2 ** attempt)))
        elif status >= 500 or (status == 409 and not path.startswith("/snapshots/")):
            time.sleep(2 ** attempt)
        elif status == 409:
            # On a snapshot call, 409 means the snapshot isn't open: it was completed, or a newer one replaced it. That
            # never changes, so don't retry. If the response to "complete" was lost, it may have worked: look for a
            # held snapshot under Admin > Matching, and check the source's records, before you run this again.
            raise RuntimeError(f"409 {response.text}: the snapshot isn't open. It may already have completed.")
        elif status >= 400:
            raise RuntimeError(f"{status} {response.text}")
        else:
            return response
    raise RuntimeError(f"Gave up on {method} {path}")


def send_snapshot(records):
    """records: every record the source holds, as a list of dicts."""
    snapshot_id = send("POST", "/snapshots").json()["snapshotId"]
    for start in range(0, len(records), 1000):
        send("POST", f"/snapshots/{snapshot_id}/records", json=records[start:start + 1000])
    return send("POST", f"/snapshots/{snapshot_id}/complete").json()


result = send_snapshot([
    {"sourceRecordId": "CRM-10042", "firstName": "Priya", "lastName": "Sharma", "email": "priya.sharma@example.com"},
    {"sourceRecordId": "CRM-10043", "firstName": "Tom", "lastName": "Fletcher", "email": "tom.fletcher@example.com"},
])
print(result)  # for example {'status': 'Applied', 'missingCount': 3}
if result["status"] == "Held":
    print("An admin has to decide on Admin > Matching before anything is deleted.")

If your list comes from a file

An admin can upload the file as a complete list in the app: it works the same way, with the same 10% hold. Upload a CSV file has the steps.