Dowser.OpenSearch.Codec (Dowser.OpenSearch v0.4.0)

View Source

Casts OpenSearch documents and search results to and from native Elixir terms, based on each document's own index mapping — a Dowser.Client :decoder and :encoder:

config :my_app, MyApp.Search,
  endpoint: "http://localhost:9200",
  decoder: Dowser.OpenSearch.Codec,
  encoder: Dowser.OpenSearch.Codec

Once configured, every API function in Dowser.OpenSearch.Document and Dowser.OpenSearch.Search casts automatically — no per-call option needed — and a call that passes :struct or :mapping casts even on a repo without it. Mappings are fetched (and cached) through Dowser.OpenSearch.MappingCacher, which the application starts.

The casting itself is dowser_client's, shared with the other backend: Dowser.Codec documents the options and what happens when a mapping can't be fetched, Dowser.Source how a document is walked, and Dowser.Type how each value is cast. This module is OpenSearch's configuration of it:

  • mappings come from Dowser.OpenSearch.MappingCacher unless :mapping says otherwise;
  • :types, :unknown_fields and :mapping_failure default to the :dowser_opensearch application environment (config :dowser_opensearch, types: MyApp.Types, mapping_failure: :warn);
  • a failed fetch emits [:dowser_opensearch, :mapping, :failure];
  • a flat_object field is opaque — see below.

Subtrees the mapping declares opaque

A flat_object field (OpenSearch's name for Elasticsearch's flattened), and an object mapped "enabled": false, hold whatever the document put in them — the mapping enumerates none of it. Both are returned exactly as they arrived: values uncast, and keys left as strings rather than run through opts[:key_fn].

That last part matters under keys: :atoms. Atoms are never garbage collected and the table is capped (:erlang.system_info(:atom_limit), a little over a million by default); atomizing keys that come from document content rather than from the mapping turns any writer into a way to exhaust it and bring the node down. A mapped field's name is one of a finite set, so casting it is safe; a flat_object field's keys are not.

A mixed result is the price: %{title: "hi", roster: %{"Managed Care Biller" => "Sam"}}. The alternative is a cast that is unsafe by construction.

Note that this is narrower than the whole risk. keys: :atoms also casts keys the mapping simply doesn't mention — an unmapped field under "dynamic": false, or one added since the mapping was cached. Where a body is wholly untrusted, keys: :atoms! (String.to_existing_atom/1) is the option that cannot grow the table at all.

The casting options

Five options decide how documents are cast; each can be passed on any API call, given to the codec in a repo's configuration, or set for the whole application — most specific first:

Dowser.OpenSearch.Search.search(query, index: "posts", types: MyApp.Types)

config :my_app, MyApp.Search,
  decoder: {Dowser.OpenSearch.Codec, types: MyApp.Types},
  encoder: {Dowser.OpenSearch.Codec, types: MyApp.Types}

config :dowser_opensearch, types: MyApp.Types
  • :mapping — where the mapping comes from: a map, a Dowser.Mapping module, or a document module. Defaults to Dowser.OpenSearch.MappingCacher, the cluster's mapping. Documents come back as maps.
  • :struct — a document module (Dowser.Schema): every document is loaded into its struct, and its struct dumped back, against its own mapping. Per call only.
  • :types — the Dowser.Type module leaf values are cast through (default Dowser.Type); see Dowser.Type for writing one.
  • :unknown_fields — :keep (default) or :raise a Dowser.UnknownFieldError for a field the mapping doesn't declare.
  • :mapping_failure — :error (default), :warn or :ignore when the mapping can't be fetched; see Dowser.Codec.

:mapping and :struct ask for casting outright: on a call to a repo with no :decoder/:encoder configured, they bring this codec in for that call.

Summary

Functions

Casts every document found in body, returning the decoded term; see Dowser.Codec.decode/3.

Casts source against the mapping of opts[:index], returning the encoded term; see Dowser.Codec.encode/3.

Functions

decode(body, opts)

@spec decode(term(), keyword()) :: term()

Casts every document found in body, returning the decoded term; see Dowser.Codec.decode/3.

encode(source, opts)

@spec encode(term(), keyword()) :: term()

Casts source against the mapping of opts[:index], returning the encoded term; see Dowser.Codec.encode/3.

encode_bulk(operations, fun, opts)

@spec encode_bulk([map()], (term(), keyword() -> term()), keyword()) :: [map()]

Casts the payloads of a Dowser.OpenSearch.Document.bulk/2 operation list; see Dowser.Codec.encode_bulk/3.