# `ExDataSketch.Storage.DETS`
[🔗](https://github.com/thanos/ex_data_sketch/blob/main/lib/ex_data_sketch/storage/dets.ex#L1)

DETS-backed persistence for sketches.

Stores serialized EXSK v2 frames in a DETS table. DETS provides disk-backed
storage that survives process and node restarts, at the cost of slower
operations compared to ETS.

## Table Requirements

- The DETS table must be `:set` type. `:ordered_set` is not supported by DETS.
- The table must be opened by the caller before use.
- DETS has a 2GB file size limit.

## Concurrency

- Individual `:dets.insert/2`/`:dets.lookup/2` calls are each atomic.
- `merge/3`'s read-modify-write cycle is **not** atomic across concurrent
  writers -- nothing holds a lock across the read and the write, so two
  concurrent `merge/3` calls on the same key can race and lose one merge,
  identically to `ExDataSketch.Storage.ETS`. See `merge/3`'s own
  documentation for the empirically-verified detail.

## Examples

   # Open a DETS table (application concern)
   {:ok, _} = :dets.open_file(:sketches, [type: :set])

   # Save a sketch
   :ok = ExDataSketch.Storage.DETS.save(sketch, :sketches, "cardinality:2024-01")

   # Load a sketch
   {:ok, sketch} = ExDataSketch.Storage.DETS.load(ExDataSketch.HLL, :sketches, "cardinality:2024-01")

   # Close when done
:ok = :dets.close(:sketches)

# `delete`

```elixir
@spec delete(atom(), ExDataSketch.Storage.key()) :: :ok | {:error, term()}
```

Deletes a sketch from the DETS table by key.

## Arguments

- `table` -- the DETS table name (atom).
- `key` -- the key to delete.

## Returns

- `:ok` on success (including when the key did not exist).
- `{:error, reason}` if the DETS operation fails.

## Examples

    iex> :dets.open_file(:test_dets_del, type: :set)
    iex> sketch = ExDataSketch.HLL.new(p: 10)
    iex> ExDataSketch.Storage.DETS.save(sketch, :test_dets_del, "hll:test")
    :ok
    iex> ExDataSketch.Storage.DETS.delete(:test_dets_del, "hll:test")
    :ok
    iex> ExDataSketch.Storage.DETS.load(ExDataSketch.HLL, :test_dets_del, "hll:test")
    {:error, :not_found}
    iex> :dets.close(:test_dets_del)
    :ok

# `load`

```elixir
@spec load(module(), atom(), ExDataSketch.Storage.key()) ::
  {:ok, struct()} | {:error, :not_found | term()}
```

Loads a sketch from the DETS table by key.

The binary value is deserialized using the given sketch module's
`deserialize/1` function.

## Arguments

- `sketch_module` -- the sketch module atom (e.g., `ExDataSketch.HLL`).
- `table` -- the DETS table name (atom).
- `key` -- the key to look up.

## Returns

- `{:ok, sketch}` on success.
- `{:error, :not_found}` if the key does not exist.
- `{:error, %DeserializationError{}}` if the stored binary is corrupted.
- `{:error, reason}` on DETS or other deserialization failures.

## Examples

    iex> :dets.open_file(:test_dets_load, type: :set)
    iex> sketch = ExDataSketch.HLL.new(p: 10) |> ExDataSketch.HLL.update("a")
    iex> ExDataSketch.Storage.DETS.save(sketch, :test_dets_load, "hll:test")
    :ok
    iex> {:ok, loaded} = ExDataSketch.Storage.DETS.load(ExDataSketch.HLL, :test_dets_load, "hll:test")
    iex> ExDataSketch.HLL.estimate(loaded) > 0.0
    true
    iex> ExDataSketch.Storage.DETS.load(ExDataSketch.HLL, :test_dets_load, "nonexistent")
    {:error, :not_found}
    iex> :dets.close(:test_dets_load)
    :ok

# `merge`

```elixir
@spec merge(struct(), atom(), ExDataSketch.Storage.key()) :: :ok | {:error, term()}
```

Merges a sketch into the persisted value at the given key.

If no sketch exists at the key, this is equivalent to `save/3`. Otherwise,
the persisted sketch is loaded, merged with the given sketch, and saved back.

**Warning:** like `ExDataSketch.Storage.ETS`, this is not atomic under
concurrent writers -- `:dets.lookup/2` and `:dets.insert/2` are each
atomic individually, but nothing holds a lock across the read and the
write, so two processes calling `merge/3` on the same key concurrently
can still race and lose one merge (empirically verified: 50 concurrent
merges of distinct elements into one key undercounted identically to
ETS). For atomic merge guarantees, use `ExDataSketch.Storage.Mnesia`,
`ExDataSketch.Storage.CubDB`, or `ExDataSketch.Storage.Ecto`, all of
which wrap the read-modify-write cycle in a real transaction.

## Arguments

- `sketch` -- the sketch to merge into the persisted value.
- `table` -- the DETS table name (atom).
- `key` -- the key whose persisted sketch to merge into.

## Returns

`:ok` on success, or `{:error, reason}` on failure.

## Examples

    iex> :dets.open_file(:test_dets_merge, type: :set)
    iex> sketch_a = ExDataSketch.HLL.new(p: 10) |> ExDataSketch.HLL.update("a")
    iex> ExDataSketch.Storage.DETS.save(sketch_a, :test_dets_merge, "hll:test")
    :ok
    iex> sketch_b = ExDataSketch.HLL.new(p: 10) |> ExDataSketch.HLL.update("b")
    iex> ExDataSketch.Storage.DETS.merge(sketch_b, :test_dets_merge, "hll:test")
    :ok
    iex> {:ok, merged} = ExDataSketch.Storage.DETS.load(ExDataSketch.HLL, :test_dets_merge, "hll:test")
    iex> ExDataSketch.HLL.estimate(merged) >= 1.9
    true
    iex> :dets.close(:test_dets_merge)
    :ok

# `save`

```elixir
@spec save(struct(), atom(), ExDataSketch.Storage.key()) :: :ok | {:error, term()}
```

Persists a sketch under the given key in the DETS table.

The sketch is serialized to an EXSK v2 binary frame before storage.

## Arguments

- `sketch` -- a sketch struct.
- `table` -- the DETS table name (atom).
- `key` -- the key under which to store the sketch.

## Returns

`:ok` on success, or `{:error, reason}` on failure.

## Examples

    iex> :dets.open_file(:test_dets_save, type: :set)
    iex> sketch = ExDataSketch.HLL.new(p: 10) |> ExDataSketch.HLL.update("a")
    iex> ExDataSketch.Storage.DETS.save(sketch, :test_dets_save, "hll:test")
    :ok
    iex> :dets.close(:test_dets_save)
    :ok

---

*Consult [api-reference.md](api-reference.md) for complete listing*