Public library (sparkth.lib)¶
sparkth/lib/ is the curated, stable API that application code and plugins import
from, instead of reaching into sparkth.core.* (or sparkth.llm.* / sparkth.rag.*)
directly (see #379). Everything below is
generated from the module docstrings.
The plugin authoring surface and the permissions API have their own pages: Plugin authoring and Permissions.
Database sessions¶
sparkth.lib.db ¶
Database session access for Sparkth — the curated public session API.
This is the single public entry point for obtaining a database session. All code,
including plugins, should acquire sessions from here rather than reaching for the
raw SQLAlchemy engine in :mod:sparkth.core.db or :mod:sparkth.core.analytics.db.
The engine itself stays in :mod:sparkth.core.db (app DB) and :mod:sparkth.core.analytics.db
(analytics DB); this module is only the public face that hands out sessions over them.
analytics_session_scope
async
¶
analytics_session_scope(
expire_on_commit: bool = False,
) -> AsyncGenerator[SQLAlchemyAsyncSession, None]
Open an async session against the analytics database as a managed context.
Identical in contract to :func:session_scope, but bound to the analytics
engine (the separate analytics database). Use this for background/non-request
analytics code (event ingestion, rollup maintenance). Inside request handlers,
prefer the :func:get_analytics_session dependency.
Delegates to analytics_db.open_analytics_session — the same seam that the
test suite overrides to inject a throwaway engine.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
expire_on_commit
|
bool
|
Whether ORM objects are expired after |
False
|
Yields:
| Type | Description |
|---|---|
AsyncGenerator[AsyncSession, None]
|
A SQLAlchemy :class: |
AsyncGenerator[AsyncSession, None]
|
analytics async engine. |
get_analytics_session
async
¶
get_analytics_session() -> AsyncGenerator[
SQLAlchemyAsyncSession, None
]
FastAPI dependency providing a SQLAlchemy AsyncSession bound to the analytics database.
Delegates to :func:analytics_session_scope. Parameterless for the same
reason as :func:get_async_session.
get_async_session
async
¶
get_async_session() -> AsyncGenerator[AsyncSession, None]
FastAPI dependency that provides an :class:AsyncSession to async def handlers.
Delegates to :func:session_scope. It is intentionally parameterless:
FastAPI turns a dependency's parameters into request (query) parameters, so
the expire_on_commit knob must not be exposed here — use
:func:session_scope directly when you need to override it.
session_scope
async
¶
session_scope(
expire_on_commit: bool = False,
) -> AsyncGenerator[AsyncSession, None]
Open an async database session as a managed context.
Yields an :class:AsyncSession — a unit-of-work that borrows a connection
from the shared engine (sparkth.core.db.get_engine), tracks the ORM objects you load and
mutate, and returns the connection to the pool when the async with block
exits (whether normally or via an exception). Always use it as a context
manager so the connection is never leaked::
from sparkth.lib.db import session_scope
async with session_scope() as session:
session.add(obj)
await session.commit()
Use this for code that runs outside an HTTP request — background tasks,
plugin bootstrap, CLI jobs, cleanup routines — where FastAPI's dependency
injection is not available. Inside request handlers, prefer the
:func:get_async_session dependency instead.
Transaction semantics: the caller is responsible for committing
(await session.commit()); any un-committed work is rolled back when the
context exits.
The expire_on_commit parameter defaults to False, which is the
async-safe choice. By default SQLAlchemy expires every ORM object after
commit(), so the next attribute access lazily re-issues a SELECT to
reload it. In synchronous code that reload is a transparent (if wasteful)
blocking query, but in async code it is implicit I/O that cannot be awaited
and fails once the session has closed. Keeping expire_on_commit=False
leaves already-loaded attributes valid after the commit and after the block.
Pass expire_on_commit=True only if you specifically want post-commit
objects to refresh on next access.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
expire_on_commit
|
bool
|
Whether ORM objects are expired after |
False
|
Yields:
| Name | Type | Description |
|---|---|---|
An |
AsyncGenerator[AsyncSession, None]
|
class: |
Logging¶
sparkth.lib.log ¶
Centralized logging for Sparkth.
The single public entry point for logging. All modules — application code and
plugins alike — must obtain loggers via :func:get_logger, never
logging.getLogger directly. Logging is configured exactly once per process
via :func:configure_logging, which is the only logging.basicConfig call in
the codebase.
Example
from sparkth.lib.log import get_logger
logger = get_logger(__name__)
logger.info("This is a log message")
configure_logging ¶
configure_logging(level: int = logging.INFO) -> None
Configure root logging once for the whole process.
This is the only place in the codebase that calls logging.basicConfig.
It is idempotent: basicConfig is a no-op if the root logger already has
handlers, so calling it from multiple entrypoints is safe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
level
|
int
|
Root logging level (defaults to |
INFO
|
get_logger ¶
get_logger(name: str) -> logging.Logger
Return the Sparkth logger for name (typically __name__).
Loggers are namespaced under sparkth and propagate to the root logger
configured by :func:configure_logging. Application modules pass __name__,
which already starts with sparkth (the package name), so it is used as-is;
any other name is prefixed with sparkth. to keep every logger under the
single sparkth root.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Logger name, typically |
required |
Returns:
| Type | Description |
|---|---|
Logger
|
Logger instance. |
Settings¶
sparkth.lib.settings ¶
Application settings public API for Sparkth.
The single public entry point for application settings. All modules — application
code and plugins alike — must access settings via :func:get_settings, never by
importing from sparkth.core.config directly.
A plugin defining its own BaseSettings class must pass :data:ENV_FILES as
its env_file so it reads the same env files, in the same precedence order,
as the core Settings class.
Example
from sparkth.lib.settings import get_settings
settings = get_settings()
print(settings.SECRET_KEY)
Language¶
sparkth.lib.language ¶
Public API for language resolution and language naming.
The single entry point for the supported-language allowlist, for checking whether a
language tag belongs to it, and for naming an arbitrary language tag for a prompt.
Application code and plugins import from here, never from sparkth.core.language or
sparkth.core.config.
Example
from sparkth.lib.language import is_supported_language, language_display_name
if current_user.language and is_supported_language(current_user.language):
... # membership check, e.g. before binding
name = language_display_name("pt-BR") # "Portuguese (Brazil)", for a prompt
LanguageInfo ¶
Bases: NamedTuple
Display names for a supported language.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
The language's name in English, for logs and admin surfaces. |
native_name |
str
|
The endonym, for the user-facing picker — someone looking for their language reads it in their own language. |
is_supported_language ¶
is_supported_language(tag: str) -> bool
Whether tag is one of the platform's supported BCP 47 tags.
The single expression of allowlist membership: DEFAULT_LANGUAGE validation
and every other caller share it, so the rule cannot drift between what the
platform accepts as its default and what it accepts from a user. Matching is
exact and case-sensitive — en-US and EN are unsupported rather than
normalised to en.
language_display_name ¶
language_display_name(tag: str | None) -> str
The English name of the language tag identifies.
tag is an arbitrary BCP 47 tag supplied by a caller — it is deliberately not
checked against :data:SUPPORTED_LANGUAGES, which governs the interface
translations the platform ships rather than the languages the model may write in.
"de" therefore yields "German" even though no German interface exists.
Falls back to the platform default's name when tag is absent, is not a
parseable language tag, or parses to a locale with no English display name in
CLDR (e.g. "skr", Saraiki) — so a misspelled or obscure value degrades to the
default instead of failing the caller's whole request or surfacing the literal
string "None". Babel's parser defaults to the POSIX underscore separator, so
the BCP 47 hyphen is passed explicitly.
resolve_language ¶
resolve_language(tag: str | None) -> str
The language a stored preference of tag resolves to.
tag is a user's stored choice, or None when they never made one.
Returns it when it is still supported, and otherwise DEFAULT_LANGUAGE —
which covers both cases where the stored value cannot be honoured: the user
never chose, and the tag has since left the allowlist. A language withdrawn
from the list (say, because its shipped interface translation was pulled)
must stop being handed back for the users who had already picked it.
Internationalization¶
sparkth.lib.i18n ¶
Static-translation (i18n) public API for Sparkth.
The single public entry point for translating user-facing strings. All modules
— application code and plugins alike — must import the marking functions from
here, never from sparkth.core.i18n directly.
How to mark a string:
-
Inside request handling, wrap the literal in :func:
gettext, imported as_::from sparkth.lib.i18n import _
raise HTTPException(status_code=401, detail=_("Incorrect username or password"))
-
f-strings cannot be extracted by
pybabel; convert them tostr.formaton the translated template::_("Role not found: {role_name}").format(role_name=role_name)
-
Module-level constants evaluate at import, before any request locale exists; mark them with :func:
lazy_gettextand render withstr()at the boundary::from sparkth.lib.i18n import lazy_gettext
GREETING_MESSAGE = lazy_gettext("Hello! How can I help you?")
-
A constant stored in a plain-
strfield (a dataclass a response model embeds, where a :class:LazyStringcannot go): mark the literal with :func:gettext_noopat the definition and translate the stored value by passing it through :func:gettextwhere it is rendered::DisplayInfo(gettext_noop("Create Course"), gettext_noop("Build courses with AI"))
Extraction and catalogs are driven by the i18n.* Make targets (see
make help). Core's catalogs live in sparkth/locale; a plugin ships its
own by registering its catalog directory on the :data:LOCALE_DIRS hook from
its __init__::
from pathlib import Path
from sparkth.lib.i18n import LOCALE_DIRS
LOCALE_DIRS.add_item(Path(__file__).parent / "locale")
The locale itself is negotiated per request from Accept-Language by the
locale middleware and read via :func:get_locale. The allowlist that
negotiation matches against, and the resolution of a user's stored preference,
belong to :mod:sparkth.lib.language; this module covers translation only.
LazyString ¶
LazyString(message: str)
A translation deferred to render time.
Module-level constants evaluate at import, before any request locale
exists; wrapping them keeps the marking at the definition site while the
translation happens when the value is rendered. The object is not a
str — call str() on it at the boundary (response serialization,
message dispatch).
bind_locale ¶
bind_locale(tag: str) -> None
Install tag as the active locale for the rest of the request.
Mirrors :func:sparkth.core.audit.context.bind_audit_actor: the locale middleware
runs before authentication, so it can only negotiate Accept-Language. Once a
signed-in user is resolved, their stored choice is the better answer and replaces the
negotiated one — they chose it, and their browser did not.
Deliberately not a context manager: there is nothing to restore, because the request's
context is discarded when its task ends. This relies on the caller running in the same
task as the route, which holds for an async FastAPI dependency but not for a sync
one — a sync dependency runs in a threadpool with a copied context and the binding
would be lost.
gettext ¶
gettext(message: str) -> str
Translate message into the active request locale.
The canonical marker for user-facing strings; import it as _ so
pybabel extract picks the call sites up.
An empty message is returned as-is: a compiled catalog stores its metadata
header under the empty msgid, so looking one up would return the PO header
instead. Callers reach this with an empty string through stored source
messages (a gettext_noop-marked field left blank), not from a literal.
gettext_noop ¶
gettext_noop(message: str) -> str
Mark message for extraction and return it unchanged.
For English strings stored in plain-str fields (e.g. the frontend
metadata dataclasses serialized by Pydantic, where a :class:LazyString
cannot go). The stored source string is translated later by passing it
through :func:gettext at the rendering boundary. Extraction picks the
literals up via pybabel extract -k gettext_noop.
lazy_gettext ¶
lazy_gettext(message: str) -> LazyString
Mark message for translation but defer it to render time.
For module-level constants and other values built before a request locale
exists. Extraction picks these up via pybabel extract -k lazy_gettext.
Encryption¶
sparkth.lib.encryption ¶
Public API for the symmetric-encryption service.
Import get_encryption_service (and EncryptionService) from here instead of
reaching into sparkth.core.encryption directly. It encrypts stored secrets — such
as LLM API keys — at rest with Fernet. Implementation lives in
sparkth/core/encryption.py.
EncryptionService ¶
EncryptionService(encryption_key: str)
Encrypts and decrypts text with Fernet.
encryption_key is one Fernet key, or several separated by commas to rotate keys: the
first encrypts, and any of them decrypts.
Data models¶
sparkth.lib.models ¶
Public API for the SQLModel data models that plugins consume.
Plugins import model classes and the shared mixins from here instead of reaching
into sparkth.core.models.* directly — every internal symbol a plugin imports
becomes an implicit public API and blocks refactoring (see issue #379).
Implementation lives in sparkth/core/models/.
AnalyticsView ¶
Bases: TimestampedModel
A saved Explore configuration, private to its owner.
AuditEvent ¶
Bases: TimestampedModel, SQLModel
One audited action, carrying the NIST AU-3 six fields plus AI provenance.
canonical_bytes holds the pinned canonical serialization of the event
(see :mod:sparkth.core.audit.canonical) so the tamper-evidence sealer can
later hash exactly what was written, byte for byte. The seal deliberately
leaves out actor_label, request_ip and user_agent: they are the
personal data a GDPR erasure blanks (actor_id stays, as a pseudonym
that only the user table can resolve), and they must be erasable
without touching the sealed content.
from_event
classmethod
¶
from_event(
event: BaseAuditEvent, context: AuditContext
) -> AuditEvent
Flatten event and its origin context into a row.
The single assembly point for rows: resolves the actor (event actor,
else context actor, else anonymous) and the timestamp, redacts every
payload, pins the canonical bytes, and maps the grouped value objects
onto the flat, indexable columns, so a row can never be constructed
with columns inconsistent with one another. All construction goes
through here (via :func:sparkth.lib.audit.record_event), never the
bare field-by-field constructor.
TZDateTime ¶
Bases: TypeDecorator[datetime]
Column type for datetimes that are timezone-aware UTC on both sides.
PostgreSQL stores timestamptz natively, but SQLite drops the UTC offset on
storage, so values that read back naive are re-tagged as UTC. Naive
datetimes are rejected on write — callers must store aware values
(see :func:utc_now).
User ¶
Bases: TimestampedModel, SoftDeleteModel
Authentication¶
sparkth.lib.auth ¶
Bearer-token authentication: the token-reading helpers and the current-user dependency.
The single canonical home for get_current_user: every caller (routes, the permission
gate, plugins, and the test harness) imports it from here. It is deliberately not
re-exported from sparkth.api.v1.auth — one object keeps FastAPI dependency overrides working.
get_current_user is a FastAPI dependency, so it is only available to code that runs
inside a route. Code that must identify the caller earlier — PluginAccessMiddleware
runs before routing, so no dependency has resolved yet — composes the same two helpers this
module builds the dependency from, :func:decode_token_username and
:func:get_user_by_username, rather than decoding tokens or querying users of its own. One
implementation of "who is this request from" keeps the security gate from drifting away from
the dependency as tokens or user lookup change.
:func:current_user_id is the trusted caller identity for tool handlers. It is bound beside the
audit actor on every request that resolves get_current_user, and is unavailable on
/ai/mcp, which authenticates no one.
bind_current_user_id ¶
bind_current_user_id(user_id: int) -> None
Bind user_id as the caller current_user_id returns for the rest of this context.
Set inside the request's own context. Tasks the request spawns copy it, and the value
never leaks into the next request. Called by bind_request_user, and by tests that act
for a user outside a request.
bind_interface_locale ¶
bind_interface_locale(user: User) -> None
Install user's stored language as the request's interface locale.
Called from both of get_current_user's exits, and from PluginAccessMiddleware.
The cached path matters as much as the full lookup: the gate populates
request.state.user on every authenticated plugin route, so binding only on the lookup
path would leave the stored preference unapplied on all of them. The gate calls this
itself as well, because the 403 it renders never reaches a route for the dependency to
run — by then it has decoded the token and loaded the user, so the preference is known.
The membership check is what keeps a language withdrawn from the allowlist from being
honoured. resolve_language is deliberately not used — its fallback would overwrite
a negotiated Accept-Language with DEFAULT_LANGUAGE, which is worse than
leaving the header in charge.
bind_request_user ¶
bind_request_user(user: User) -> None
Apply everything a resolved caller implies for the rest of the request.
Installs the user's interface locale, binds them as the audit actor, so every audit event
the request records is attributed to them, and binds their id for current_user_id.
Called from both exits of get_current_user. The plugin gate binds the locale on its
own, because its 403 never reaches a route.
current_user_id ¶
current_user_id() -> int
Return the id of the user authenticated for the running request.
For tool handlers that act for a user. The id comes from the request's token, never from a tool argument, so a model cannot name another user. It is independent of the audit context, so authorization never depends on auditing.
Bound on every REST request that resolves get_current_user, including chat turns and
the tools they run on the turn's detached task. Never bound on /ai/mcp, which
authenticates no caller.
Raises:
| Type | Description |
|---|---|
NoAuthenticatedUser
|
nothing is bound, for example on |
decode_token_username ¶
decode_token_username(token: str) -> str | None
Return the username a bearer token identifies, or None if it identifies nobody.
None covers every way a token can fail to name a user — malformed, signed with the
wrong key, expired, or carrying no sub claim — because callers treat them alike:
an unreadable token is an unauthenticated request. Callers decide what that means;
this helper never raises.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
token
|
str
|
The raw JWT from the |
required |
get_current_user
async
¶
get_current_user(
request: Request,
credentials: HTTPAuthorizationCredentials = Depends(
security_scheme
),
session: AsyncSession = Depends(get_async_session),
) -> User
Resolve the authenticated user, rejecting the request when the token names no one.
Reuses the user PluginAccessMiddleware left on request.state when it has already
resolved this request's caller, rather than decoding the same token and re-reading the
same row. Only that gate writes the attribute, and only from this request's own token,
so the answer is the one this dependency would have computed. Requests it never
identified — core routes, which it does not gate — fall through to the full lookup.
The reused instance is detached: the gate's session has closed by the time the route
runs. That is safe because User maps only columns, which stay readable on a detached
instance; a test in tests/core/plugins/test_middleware.py pins that, since adding a
relationship to User is what would make this unsafe.
A resolved user's stored language replaces the locale the middleware negotiated from
Accept-Language, so translated copy follows what the user chose rather than what
their browser advertises, on both exits below — the cached one and the full lookup.
An unset or unsupported stored value leaves the negotiated locale in place.
get_user_by_username
async
¶
get_user_by_username(
username: str, session: AsyncSession
) -> User | None
Return the user with this username, or None when no user has it.
RAG¶
sparkth.lib.rag ¶
Public API for the RAG library.
All plugins and external modules import RAG functionality from here.
DocumentNotFoundError ¶
Bases: RAGError
Raised when a document does not exist or is not accessible to the user.
DocumentSection
dataclass
¶
DocumentSection(
source_name: str,
chapter: str | None,
section: str | None,
subsection: str | None,
chunk_count: int,
position_index: int,
)
RAGNotReadyError ¶
RAGNotReadyError(document_id: int, status: str)
Bases: RAGError
Raised when the document exists but status is not READY.
RAGRetrievalError ¶
Bases: RAGError
Raised when agent retrieval or section-chunk fetch fails.
RetrievedChunk
dataclass
¶
RetrievedChunk(
source_name: str,
chapter: str | None,
section: str | None,
subsection: str | None,
content: str,
)
A chunk returned by retrieval, with its document/section attribution.
ScannedPDFError ¶
ScannedPDFError(source_name: str)
Bases: RAGError
Raised when a PDF appears to be scanned/image-only.
UnsupportedFileTypeError ¶
Bases: RAGIngestionError
Raised when a file's type cannot be handled by the RAG extractors.
Nothing from the file enters the corpus, so callers should report the document as failed rather than ready.
agentic_retrieve_context
async
¶
agentic_retrieve_context(
query: str, document_ids: list[int], llm: BaseChatModel
) -> list[RetrievedChunk]
Retrieve relevant document chunks for a query across the given documents.
Validates that every document exists and is READY before retrieval — callers do not need to perform this check themselves. Uses agentic section retrieval per document and returns a flat list of RetrievedChunk. Opens its own database sessions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
document_ids
|
list[int]
|
Documents to search. All must exist and be READY. |
required |
query
|
str
|
The user's natural-language query. |
required |
llm
|
BaseChatModel
|
LangChain chat model used by the retrieval agent. |
required |
Returns:
| Type | Description |
|---|---|
list[RetrievedChunk]
|
Flat list of RetrievedChunk across all documents (empty if no matches). |
Raises:
| Type | Description |
|---|---|
DocumentNotFoundError
|
a document is missing or soft-deleted. |
RAGNotReadyError
|
a document exists but is not READY. |
RAGRetrievalError
|
retrieval failed. |
copy_document_chunk_links
async
¶
copy_document_chunk_links(
session: AsyncSession,
source_document_id: int,
target_document_id: int,
) -> None
Copy all chunk links from source_document_id to target_document_id.
format_document_chunks_as_llm_context ¶
format_document_chunks_as_llm_context(
retrieved_chunks: list[RetrievedChunk],
) -> str
Group retrieved chunks by document and render them as one LLM context string.
Retrieval can return chunks from several documents interleaved in relevance order. They are grouped by the document each chunk came from (keyed by the document name the chunk carries, in first-seen order); each document's chunks are rendered as a labelled context block, and the blocks are joined with a blank line into a single string. An empty input yields an empty string.
get_rag_ingested_document_structure
async
¶
get_rag_ingested_document_structure(
document_id: int,
) -> list[DocumentSection]
Return ordered section metadata generated from the ingested RAG chunks.
Sections are ordered by the minimum chunk id within each (chapter, section, subsection) group. This preserves document insertion order, so position_index reliably reflects the section's position in the original document.
Returns an empty list when the document does not exist.
ingest_document
async
¶
ingest_document(
filename: str, file_bytes: bytes, document_id: int
) -> IngestionResult
Ingest a document's bytes into the RAG store.
Pipeline: eligibility check -> extract -> chunk -> store (with cross-document content-hash dedup) -> link chunks to document_id. Opens and commits its own database session.
Audited as a rag.document_ingested event. A success is recorded in the
same transaction as the chunk write (fail-closed, so unrecordable content
cannot enter the corpus). A failure is recorded, in its own transaction
before the error propagates, for the two eligibility/extraction errors this
function declares below; any other error (a parse failure inside an
extractor, a chunker error, a database error from the chunk write)
propagates without an audit record. Nothing entered the corpus in those
cases, so the trail stays accurate about the corpus itself, but it is not a
complete log of attempts.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
document_id
|
int
|
Document.id recorded in the chunk-link table. |
required |
file_bytes
|
bytes
|
Raw file content. |
required |
filename
|
str
|
Original filename (drives extension dispatch). |
required |
Returns:
| Type | Description |
|---|---|
IngestionResult
|
IngestionResult with new/reused chunk counts. |
Raises:
| Type | Description |
|---|---|
UnsupportedFileTypeError
|
type the extractors cannot handle. |
ScannedPDFError
|
PDF appears scanned/image-only. |
LLM¶
sparkth.lib.llm ¶
Public API for the LLM library.
All plugins and external modules import LLM functionality from here. Nothing
outside sparkth/llm/ should import from sparkth.llm.* directly.
BaseChatProvider ¶
BaseChatProvider(
api_key: str,
model: str,
system_prompt: str | None = None,
temperature: float = 0.7,
max_tool_executions: int = 50,
max_retries: int = 2,
)
Bases: ABC
create_llm ¶
create_llm(
streaming: bool = False,
callbacks: list[Any] | None = None,
) -> Any
Public interface for creating a configured LangChain LLM instance.
send_message
async
¶
send_message(
messages: list[dict[str, Any]],
max_tokens: int | None = None,
tools: list[Any] | None = None,
) -> dict[str, Any]
Send a message and get a response, with optional tool usage.
stream_message
async
¶
stream_message(
messages: list[dict[str, Any]],
max_tokens: int | None = None,
tools: list[Any] | None = None,
) -> AsyncIterator[dict[str, Any]]
Stream a message response, with optional tool usage.
Yields typed event dicts
{"type": "token", "content": str} — text to display {"type": "tool_start","name": str} — tool execution beginning {"type": "tool_end", "name": str, "failure": ToolFailureKind | None} — tool execution finished, and how it failed
LLMConfigAdapter ¶
Base adapter for plugins that hold an optional llm_config_id reference.
preprocess_config: validates the referenced LLMConfig is owned by the user. postprocess_config: resolves llm_config_id to name/provider/model for the frontend.
LLMConfigDuplicateNameError ¶
LLMConfigDuplicateNameError(name: str)
Bases: ValueError
Raised when an LLM config with the same name already exists for the user.
LLMConfigInactiveError ¶
LLMConfigInactiveError()
Bases: ValueError
Raised when an LLM config exists but is inactive.
LLMConfigModelNotSetError ¶
Bases: ValueError
Raised when the model field is empty on an LLM config.
LLMConfigNotFoundError ¶
LLMConfigNotFoundError(config_id: int, user_id: int)
Bases: ValueError
Raised when an LLM config is not found.
LLMConfigService ¶
LLMConfigService(
encryption: SupportsEncryption, cache: SupportsCache
)
Initialize with encryption and cache services.
create
async
¶
create(
session: AsyncSession,
user_id: int,
name: str,
provider: str,
model: str,
api_key: str,
) -> LLMConfig
Create a new LLM config, encrypting the API key. Raises LLMConfigDuplicateNameError on duplicate name.
delete
async
¶
delete(
session: AsyncSession, user_id: int, config_id: int
) -> bool
Soft-delete a config and evict its cache entry. Returns False if not found.
get
async
¶
get(
session: AsyncSession, user_id: int, config_id: int
) -> LLMConfig | None
Fetch a single non-deleted LLM config by ID and user, or None if not found.
list
async
¶
list(
session: AsyncSession,
user_id: int,
include_inactive: bool = False,
) -> list[LLMConfig]
Return all non-deleted LLM configs for a user, newest first.
By default only active configs are returned. Pass include_inactive=True to also include deactivated configs (e.g. for the settings page).
mask_key
staticmethod
¶
mask_key(api_key: str) -> str
Return a masked version of the API key, preserving the provider prefix and last 4 chars.
resolve
async
¶
resolve(
session: AsyncSession, user_id: int, config_id: int
) -> tuple[LLMConfig, str]
Return (config, decrypted_api_key). Caches by (user_id, config_id). Updates last_used_at.
Every decrypted read, cache hit or miss, records an llm_config.key_read
audit event in the caller's transaction: the credential access trail. It
rides the caller's session rather than committing on its own so the read
costs no second pooled connection; both callers (chat and Slack) commit
that session right after resolving, on their error paths too, so the
record lands whenever the turn is persisted at all.
rotate_key
async
¶
rotate_key(
session: AsyncSession,
user_id: int,
config_id: int,
api_key: str,
) -> LLMConfig
Replace the API key for a config, updating encryption and invalidating cache.
set_active
async
¶
set_active(
session: AsyncSession,
user_id: int,
config_id: int,
is_active: bool,
) -> LLMConfig
Activate or deactivate a config. Bypasses the is_active filter so inactive configs can be re-activated.
update
async
¶
update(
session: AsyncSession,
user_id: int,
config_id: int,
name: str | None = None,
model: str | None = None,
) -> LLMConfig
Update name and/or model of an existing config. Raises ValueError if not found or duplicate name.
LLMConfigValidationError ¶
Bases: ValueError
Raised when a field value is invalid (e.g. model not allowed for provider).
get_provider ¶
get_provider(
provider_name: str,
api_key: str,
model: str,
system_prompt: str | None = None,
temperature: float = 0.7,
max_tool_executions: int = 50,
max_retries: int = 2,
) -> BaseChatProvider
Get a chat provider instance.
get_provider_catalog ¶
get_provider_catalog() -> list[ProviderCatalogEntry]
Return all providers with their available models.
Analytics¶
sparkth.lib.analytics ¶
Public API for the analytics emission gateway.
All application code and plugins import analytics functionality from here rather
than reaching into sparkth.core.analytics.* directly. Implementation lives in
sparkth/core/analytics/.
Plugins
- subclass
AnalyticsEventSchemato define an event payload schema (declaringevent_type/version), - register it from their
__init__viaregister_event_schema(self, MyEvent), - emit it through
emit_event, oremit_eventsfor a group of related events that should share one session (reach for the lower-levelingest_eventonly when the caller already holds an analytics session), - list the registered event types and their payload fields with
event_catalog, - measure one event type's raw events, filtered and grouped by payload fields, with
measure_raw_events(anExploreQuery; bad names or values raiseExploreQueryError), - save a user's Explore configs as private views with
create_view,list_views,get_view,update_viewanddelete_view, - read a continuous aggregate by range and bucket with
read_rollup(payload-key dims, no SQL), taking the range asRangeQuery, - add a headline figure to the Overview with
register_summary_figure(aSummaryFigurewhose count usually sumsread_rolluprows).
AnalyticsEventSchema ¶
Bases: BaseModel
Base class for analytics event payload schemas.
Subclasses set event_type and version as class attributes and declare
the payload as ordinary Pydantic fields. Those two are ClassVar — identity
metadata, not part of the validated payload — and are required: a subclass that
omits either fails at definition time (__init_subclass__).
Extra fields are forbidden so a producer sending unexpected keys gets a
422 rather than having those fields silently dropped from the stored row.
AnalyticsViewAlreadyExists ¶
Bases: Exception
The owner already has a saved view with this name.
AnalyticsViewNotFound ¶
Bases: Exception
The owner has no saved view with this id.
Bucket ¶
Bases: StrEnum
Bucket width for analytics reads: day, week or month.
ChartKind ¶
Bases: StrEnum
How an Explore result is drawn.
ContinuousAggregateNotFound ¶
ContinuousAggregateNotFound(name: str)
Bases: Exception
Raised when a backfill targets a continuous aggregate that does not exist.
DuplicateEventTypeError ¶
DuplicateEventTypeError(event_type: str, version: int)
Bases: Exception
Raised when a schema claims an already-registered (event_type, version).
Fires only for a different class claiming the identity — a startup-fatal programming error, because a producer's payload could then silently validate against the wrong schema. Re-registering the same class is a no-op, so a plugin constructed more than once does not trip this.
EventCatalogEntry ¶
Bases: BaseModel
An event type and its payload fields, merged across versions.
EventField ¶
Bases: BaseModel
One payload field of an event type. values lists the choices of an enum.
EventNamespaceError ¶
EventNamespaceError(plugin_name: str, event_type: str)
Bases: Exception
Raised when a plugin contributes an event not namespaced under its own name.
ExplorePoint ¶
Bases: BaseModel
A measure for one bucket and group. bucket is the UTC start, YYYY-MM-DD.
ExploreQuery ¶
Bases: RangeQuery
Query over one event type's raw events.
filter items are field:value (null matches a null or missing key); values of one field
are ORed, fields ANDed. group_by is a field name or actor.
ExploreQueryError ¶
Bases: Exception
Raised when an explore query names something it cannot query.
ExploreResult ¶
Bases: BaseModel
Explore points and totals; truncated when groups beyond the cap were dropped.
ExploreTotal ¶
Bases: BaseModel
A group's measure over the whole range.
FilterFieldType ¶
Bases: StrEnum
How an event payload field is filtered: it picks the filter control and how a value is parsed.
string is free text, enum one of the field's listed values, boolean true or
false, and integer a whole number.
InvalidFilterValue ¶
InvalidFilterValue(field: str, value: str)
LoginsPoint ¶
Bases: BaseModel
Logins for one bucket and sign-in method. bucket is the UTC start, YYYY-MM-DD.
Measure ¶
Bases: StrEnum
What an explore query counts per bucket.
PendingEvent
dataclass
¶
PendingEvent(
event_type: str,
version: int,
payload: dict[str, Any],
actor_id: str | None = None,
occurred_at: datetime | None = None,
)
One event waiting to be landed, for :func:emit_events.
The same values :func:emit_event takes, bundled so a caller can hand over a group
at once. occurred_at is per-event rather than per-group: events landed together
did not necessarily happen together.
RangeQuery ¶
Bases: BaseModel
Query for a bucketed read: an inclusive UTC day range and a bucket width.
SummaryCount ¶
Bases: BaseModel
A figure's count for a period and for the equal-length period before it.
SummaryFigure
dataclass
¶
SummaryFigure(
key: str,
label: str,
count: Count,
view: str | None = None,
order: int = 100,
plugin: str | None = None,
)
One headline count on the analytics Overview.
Attributes:
| Name | Type | Description |
|---|---|---|
key |
str
|
Stable id; a plugin's keys start with |
label |
str
|
|
count |
Count
|
Counts the figure for an inclusive UTC day range. |
view |
str | None
|
The analytics view that breaks it down. |
order |
int
|
Sorts ascending, then by key. |
plugin |
str | None
|
Set on registration. The count runs for every caller. |
UngroupableExploreField ¶
UngroupableExploreField(event_type: str, field: str)
Bases: ExploreQueryError
Raised when an explore query groups by a field whose kind cannot be grouped (an integer).
UnknownEventTypeError ¶
UnknownEventTypeError(event_type: str, version: int)
Bases: Exception
Raised when an (event_type, version) pair has no registered schema.
UnknownExploreEvent ¶
UnknownExploreEvent(event_type: str)
UnknownExploreField ¶
UnknownExploreField(event_type: str, field: str)
ViewConfig ¶
Bases: BaseModel
An Explore config without the period, keyed as the frontend keeps it.
ViewFilter ¶
Bases: BaseModel
One filter: the field must equal the value.
backfill_continuous_aggregates
async
¶
backfill_continuous_aggregates(
name: str | None = None,
) -> list[str] | None
Materialize the full history of continuous aggregates (all, or one by name).
Continuous aggregates are created WITH NO DATA (so creation does not backfill
inside Alembic's transaction), and their refresh policies only cover a trailing
window (start_offset). Without a one-off full refresh, buckets older than that
window fall below the materialization watermark once the first policy run advances it
and disappear from the view — so any pre-migration history is silently lost. Run this
once after applying an aggregate's migration on PostgreSQL/TimescaleDB; it is
idempotent and safe to re-run.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str | None
|
Refresh only this aggregate. When |
None
|
Returns:
| Type | Description |
|---|---|
list[str] | None
|
The list of aggregate names refreshed (possibly empty if none are registered), or |
list[str] | None
|
|
list[str] | None
|
(e.g. SQLite in tests/e2e, where continuous aggregates do not exist and the read |
list[str] | None
|
path aggregates |
Raises:
| Type | Description |
|---|---|
ContinuousAggregateNotFound
|
if |
catalog_entry ¶
catalog_entry(event_type: str) -> EventCatalogEntry | None
The catalog entry for event_type, or None when it is not registered.
create_view
async
¶
create_view(
session: AsyncSession,
owner_id: int,
name: str,
config: ViewConfig,
) -> AnalyticsView
Save a new view.
Raises:
| Type | Description |
|---|---|
AnalyticsViewAlreadyExists
|
The owner already has a view with this name. |
delete_view
async
¶
delete_view(
session: AsyncSession, owner_id: int, view_id: UUID
) -> None
emit_event
async
¶
emit_event(
event_type: str,
version: int,
payload: dict[str, Any],
actor_id: str | None = None,
occurred_at: datetime | None = None,
) -> None
Validate and land an analytics event, propagating any failure.
The producer-facing counterpart to :func:ingest_event: it opens its own
analytics session, so callers need no session plumbing. That is the only
thing it adds. It catches nothing — UnknownEventTypeError,
ValidationError, SQLAlchemyError and anything else reach the caller
unchanged, so a broken analytics write is never silently hidden.
Producers call this from FastAPI background tasks and detached asyncio tasks, which run after the response has been sent; a failure there surfaces as an unhandled task error in the logs rather than affecting the request that was being measured.
Prefer ingest_event where the caller already holds an analytics session.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
event_type
|
str
|
The registered event name, e.g. |
required |
version
|
int
|
The schema version, e.g. |
required |
payload
|
dict[str, Any]
|
The event body, validated against the registered schema. |
required |
actor_id
|
str | None
|
The acting user's id as a string, stored for provenance. |
None
|
occurred_at
|
datetime | None
|
When the event happened — producers emit from background tasks that run after the response, so a turn's events would otherwise all be stamped at the end of it and lose their order. Pass the moment the thing being recorded actually happened. Defaults to now, which is only right for an event emitted where it happened. |
None
|
Raises:
| Type | Description |
|---|---|
UnknownEventTypeError
|
No schema is registered for this type and version. |
ValidationError
|
The payload does not match the registered schema. |
SQLAlchemyError
|
The analytics database could not be reached or written. |
emit_events
async
¶
emit_events(events: Sequence[PendingEvent]) -> None
Land a group of related events, sharing one analytics session.
:func:emit_event opens a session per call, so a producer with a variable number
of related events — a completion and one event per tool it executed, say — pays
one session acquisition per event, and, because emitting them in sequence stops
at the first failure, loses every event behind a failed one.
This acquires the session once and lands the whole group in one transaction, committed once at the end. Ordering is preserved, and an empty sequence is a no-op.
The group is therefore all-or-nothing: a failure on any event — an unregistered type, an invalid payload, a database error — leaves none of them landed, so a completion never appears without the tool events that belong to it.
Catches nothing, exactly like :func:emit_event: the failure propagates to the
caller.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
events
|
Sequence[PendingEvent]
|
The events to land, in the order they should appear. |
required |
Raises:
| Type | Description |
|---|---|
UnknownEventTypeError
|
No schema is registered for one of the events. |
ValidationError
|
One payload does not match its registered schema. |
SQLAlchemyError
|
The analytics database could not be reached or written. |
event_catalog ¶
event_catalog() -> list[EventCatalogEntry]
Every event type with its fields merged across versions (the highest version wins), sorted.
get_event_schema ¶
get_event_schema(
event_type: str, version: int
) -> type[AnalyticsEventSchema]
Return the schema for (event_type, version), or raise UnknownEventTypeError.
get_logins
async
¶
get_logins(
session: AsyncSession,
from_: date,
to: date,
bucket: Bucket,
) -> list[LoginsPoint]
Return login counts per bucket and sign-in method for days from_..to (inclusive).
Oldest bucket first, methods ordered bytewise (unknown first) within a bucket. Sparse:
buckets or methods with no logins are omitted, so callers zero-fill. A login whose payload has
no method counts as unknown.
get_view
async
¶
get_view(
session: AsyncSession, owner_id: int, view_id: UUID
) -> AnalyticsView
The owner's view with this id.
Raises:
| Type | Description |
|---|---|
AnalyticsViewNotFound
|
No such view for this owner. |
ingest_event
async
¶
ingest_event(
session: AsyncSession,
event_type: str,
version: int,
payload: dict[str, Any],
actor_id: str | None = None,
occurred_at: datetime | None = None,
commit: bool = True,
) -> None
Validate payload against the registered schema and land it in raw_events.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
session
|
AsyncSession
|
An async session bound to the analytics database. |
required |
event_type
|
str
|
The base event name, e.g. |
required |
version
|
int
|
The schema version, e.g. |
required |
payload
|
dict[str, Any]
|
The raw event body to validate. |
required |
actor_id
|
str | None
|
The authenticated user id, stored for provenance. |
None
|
occurred_at
|
datetime | None
|
When the event happened; defaults to |
None
|
commit
|
bool
|
Whether to commit the insert. Pass |
True
|
Raises:
| Type | Description |
|---|---|
UnknownEventTypeError
|
No schema is registered for |
ValidationError
|
The payload does not satisfy the schema. |
SQLAlchemyError
|
The insert failed. |
list_views
async
¶
list_views(
session: AsyncSession, owner_id: int
) -> list[AnalyticsView]
The owner's views, by name.
measure_raw_events
async
¶
measure_raw_events(
session: AsyncSession, query: ExploreQuery
) -> ExploreResult
Measure query.event_type per bucket and group, keeping the top GROUP_LIMIT groups.
Totals come from a separate unbucketed query, so distinct_actors totals are distinct over
the range. Groups rank by total, then name; points are sorted by bucket then group, None
first. actor_names is left empty.
Raises:
| Type | Description |
|---|---|
UnknownExploreEvent
|
The event type is not registered. |
UnknownExploreField
|
A filter or |
InvalidFilterValue
|
A filter value does not fit its field's kind. |
UngroupableExploreField
|
|
read_rollup
async
¶
read_rollup(
session: AsyncSession,
*,
aggregate: str,
event_types: Sequence[str],
dims: Mapping[str, str | Mapping[str, str]],
by_event_type: bool = False,
count_column: str,
sums: Mapping[str, str] | None = None,
from_: date,
to: date,
bucket: Bucket,
) -> list[RowMapping]
Count events per bucket (and dims) for days from_..to (inclusive).
PostgreSQL sums count_column of the continuous aggregate aggregate, grouped by
the aggregate's columns named like the dims keys; SQLite counts raw_events of
event_types, extracting each dim from the payload. The two agree only if dims
matches the aggregate's definition, count_column is the aggregate's count, and each
sums output name is an aggregate column summing that payload key. The aggregate must
have a day bucket column, and each dim must be a string-valued payload key.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
aggregate
|
str
|
The continuous aggregate PostgreSQL reads. |
required |
event_types
|
Sequence[str]
|
The event types SQLite counts. |
required |
dims
|
Mapping[str, str | Mapping[str, str]]
|
Output column -> payload key, either one key for every event type or a per-event-type mapping (NULL for event types it omits). Payload keys are never SQL. |
required |
by_event_type
|
bool
|
Also group by |
False
|
count_column
|
str
|
The aggregate's count column, also the output count's name. |
required |
sums
|
Mapping[str, str] | None
|
Output column -> payload key to sum; PostgreSQL sums the aggregate's column of that output name. Each sum is an integer, 0 when there is nothing to sum. |
None
|
Returns:
| Type | Description |
|---|---|
list[RowMapping]
|
Rows of |
list[RowMapping]
|
and |
list[RowMapping]
|
first. Sparse. |
Raises:
| Type | Description |
|---|---|
ValueError
|
A name, payload key or event type interpolated into SQL is unsafe. |
register_event_schema ¶
register_event_schema(
plugin: SparkthPlugin,
schema: type[AnalyticsEventSchema],
) -> None
Register a plugin's event schema on the ANALYTICS_EVENTS hook.
Call this from a plugin's __init__. Registration happens at
import time, straight into the ANALYTICS_EVENTS hook the gateway resolves
against.
Two startup-fatal guards, enforced here so a misconfigured plugin crashes the
process at import rather than at first emit (a third — that the schema declares
event_type/version — is enforced on AnalyticsEventSchema itself, at
class-definition time, via __init_subclass__):
- Namespace.
event_typemust be prefixed with the contributing plugin's name (e.g. pluginslack→"slack.*"), elseEventNamespaceError. This stops a plugin squatting a core or another plugin's event name. - Collision. A different class claiming an already-registered
(event_type, version)raisesDuplicateEventTypeError. Re-registering the identical class is a no-op, so constructing a plugin more than once — a module re-import, or a test building its own instance — is not a collision. This followsKeyedClassHook's rule for class registries; only a different class can squat a name, which is what the guard is for.
register_event_schemas ¶
register_event_schemas(
plugin: SparkthPlugin, module: ModuleType
) -> None
Register every event schema module defines, in one call.
Each schema goes through :func:register_event_schema, so the namespace and
collision guards apply unchanged.
register_summary_figure ¶
register_summary_figure(
plugin: SparkthPlugin, figure: SummaryFigure
) -> None
Add a plugin's figure to the analytics Overview, tagged with the plugin's name.
Call from the plugin's __init__ with a figure built once at module level. Registering an
equal figure again is a no-op; a different figure under a taken key raises ValueError.
summarize
async
¶
summarize(
session: AsyncSession, from_: date, to: date
) -> list[SummaryCount]
Count every registered figure for from_..to and for the same number of days before.
Labels are translated into the request locale. Ordered by order, then key.
update_view
async
¶
update_view(
session: AsyncSession,
owner_id: int,
view_id: UUID,
name: str | None,
config: ViewConfig | None,
) -> AnalyticsView
Rename a view and/or replace its config; None leaves that part as is.
Raises:
| Type | Description |
|---|---|
AnalyticsViewNotFound
|
No such view for this owner. |
AnalyticsViewAlreadyExists
|
The new name is taken by another of the owner's views. |
HTTP client¶
sparkth.lib.http ¶
BaseHttpClient ¶
BaseHttpClient(base_url: str, auth: Auth = Auth.BEARER)
Shared base for HTTP API clients that authenticate with a token.
Subclasses provide a token property and call _request() instead of
duplicating the token-check + URL-join logic.
The optional base_url override on _request supports clients whose
verb methods accept a per-call base URL (e.g. OpenEdxClient).
Initialise the client with a base URL and authentication scheme.
token
property
¶
token: str | None
Return the bearer token for the current session; override in subclasses.
__aexit__
async
¶
__aexit__(
_exc_type: type[BaseException] | None,
_exc_val: BaseException | None,
_exc_tb: TracebackType | None,
) -> None
Close the underlying session on context manager exit.
Documents¶
sparkth.lib.documents ¶
Public API for the Document registry.
All plugins and external modules manage document identity and status from here.
Document ¶
Bases: TimestampedModel, SoftDeleteModel
Plugin-agnostic registry of documents submitted for RAG ingestion.
Plugins create one row per source document they want to ingest. RAG uses document_id as its primary reference — it never imports from any plugin model.
create_document
async
¶
create_document(
session: AsyncSession,
user_id: int,
name: str,
mime_type: Optional[str],
) -> Document
Create a new Document row with QUEUED status and flush (no commit).
get_document
async
¶
get_document(
session: AsyncSession, document_id: int, user_id: int
) -> Optional[Document]
Return the Document if owned by user_id and not soft-deleted; else None.
list_ready_documents
async
¶
list_ready_documents(
session: AsyncSession, user_id: int
) -> list[Document]
Return non-deleted READY documents owned by user_id, ordered for display.
soft_delete_document
async
¶
soft_delete_document(
session: AsyncSession, document_id: int
) -> None
Soft-delete a Document by id. Does not commit.
Records a rag.document_deleted audit event in the caller's transaction,
so the deletion and its record commit or roll back together. A missing id
is a no-op and records nothing.
Notifies :data:~sparkth.core.documents.hooks.DOCUMENT_DELETED before flushing, so a
plugin holding rows that reference the document can clear them in this same
transaction. The FK cascades declare ondelete="CASCADE", but a soft delete is an
UPDATE, so nothing else clears them (issue #702).
update_document_status
async
¶
update_document_status(
session: AsyncSession,
document_id: int,
status: DocumentStatus,
error: Optional[str] = None,
) -> None
Update status (and optional error) on an existing Document. Does not commit.
Enums¶
sparkth.lib.enums ¶
Exceptions¶
sparkth.lib.exceptions ¶
Legacy HTTP-carrying exceptions, kept for existing LMS-client callers.
These classes predate the exception→HTTP mapping standard (CLAUDE.md,
"Domain exceptions → HTTP responses"): they carry status_code on the exception
itself. New code must NOT follow this pattern — raise an HTTP-agnostic domain exception
and map it to a status via register_exception_handler (sparkth.lib.exceptions.handlers)
instead.
Cache¶
sparkth.lib.cache ¶
Shared cache for plugins: the Redis-backed :class:CacheService and its accessor.
CacheService ¶
CacheService(redis_url: str, default_ttl: int = 3600)
Async Redis cache of string values, each stored with a TTL; it connects on first use.
delete
async
¶
delete(key: str) -> bool
Remove key and return True if a value was removed, False if none was stored.
A Redis error is logged and swallowed, and False is returned.
get
async
¶
get(key: str) -> str | None
Return the value stored at key, or None when it is missing.
A Redis error is logged and swallowed, and None is returned.
make_key ¶
make_key(*parts: str) -> str
Join parts with : into one key: make_key("openedx", "7") is "openedx:7".
set
async
¶
set(key: str, value: str, ttl: int | None = None) -> bool
Store value at key for ttl seconds and return True.
A ttl of None or 0 falls back to the default TTL. A Redis error is logged and
swallowed, and False is returned.
get_cache_service
cached
¶
get_cache_service(
redis_url: str, default_ttl: int = 3600
) -> CacheService
Return the shared CacheService; calls with the same arguments get the same instance.