diff --git a/README.md b/README.md index 0d5ba5e..abcfbf6 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,7 @@ a plugin set every part of which still has somewhere to talk to. See - **Recipes in YAML.** A job is a file, not a program. No Ruby is written to wire a pipeline together. -- **36 plugins** across seven categories: subscribe, custom feed, filter, +- **41 plugins** across seven categories: subscribe, custom feed, filter, store, provide, notify, publish — and every one of them has a current use. - **Markdown out of the box.** `PublishMarkdown` writes the result as a plain Markdown document, to a file or to standard output, with no service and no @@ -374,14 +374,14 @@ like a shipped plugin replaces it. ### Which plugins still work -36 plugins ship with the gem. Every one is classified in +41 plugins ship with the gem. Every one is classified in [`doc/PLUGINS.md`](doc/PLUGINS.md) section 6, with its settings and the reason for its status: | Status | Count | Meaning | | --- | --- | --- | -| **Supported** | 25 | Works on the supported Rubies with current dependencies | -| **Supported (external)** | 10 | Works, but needs something you provide: a service, a command, a credential, a data file | +| **Supported** | 26 | Works on the supported Rubies with current dependencies | +| **Supported (external)** | 14 | Works, but needs something you provide: a service, a command, a credential, a data file | | **Needs rework** | 1 | The service exists; this plugin speaks a replaced interface | Eleven plugins were removed in this release rather than kept as history: each diff --git a/doc/DEPLOYMENT.md b/doc/DEPLOYMENT.md index 1c34a57..c8319af 100644 --- a/doc/DEPLOYMENT.md +++ b/doc/DEPLOYMENT.md @@ -471,6 +471,7 @@ bundle. `plugins` is every group in the first block at once. | `PublishInstapaper` | an Instapaper account; no gem | — | — | Supported (external) | | `PublishEject` | the `eject` or `drutil` command | — | — | Supported (external) | | `NotifyIkachan` | an `ikachan` gateway you run | — | — | Supported (external) | +| `FilterOpenAI`, `FilterClaude`, `FilterGemini`, `FilterSakuraAI` | an account and an API token with that one service; no gem | — | — | Supported (external) | | `PublishHatenaBookmark` | the current Hatena API, which it does not speak | — | — | Needs rework | The `plugins` group is the first five rows: the optional gems of the plugins diff --git a/doc/PLUGINS.md b/doc/PLUGINS.md index 42d9cb5..d242117 100644 --- a/doc/PLUGINS.md +++ b/doc/PLUGINS.md @@ -1045,6 +1045,279 @@ pipeline expects. Needed because GitHub publishes Atom, not RSS. No settings. A field that is already a string is taken as it stands, so a pipeline that has been through another filter first is no longer a `NoMethodError`. +#### FilterJoin — **Supported** + +`filter/join.rb`. Joins every item in the pipeline into one item. Many items +in, one item out, and that is the whole of it: it fetches nothing, summarizes +nothing, and knows nothing about what reads the result. + +| Key | Type | Meaning | +| --- | --- | --- | +| `title` | string | The title of the joined item. Default `Joined items`. | + +**In**: the pipeline as it stands — any number of feeds, any number of items; a +feed that is `nil` is passed over. **Out**: one feed holding one item. The whole +pipeline becomes one item rather than one item per feed, because the point of +joining is to have a single text; a Recipe that wants one item per feed still +has its feeds separate before this plugin runs. + +The description is plain text, with a numbered heading per item so that +whatever reads it can tell one article from the next: + +```text +ARTICLE 1 +Title: Ruby 4.1 released +URL: https://example.com/a + +The body of the first article. + +ARTICLE 2 +Title: PostgreSQL 19 released +URL: https://example.com/b + +The body of the second article. +``` + +A title, link or description an item does not carry is written as empty, so +every section has the same shape. **An input with no items produces an empty +pipeline**, not an item that says nothing. + +**The joined item has no link.** It is several articles at once, so there is no +page it points at, and putting the first article's URL there would name a +source for text that is not only from it. That has one consequence for a +Recipe: the store plugins are keyed on the link and drop an item without one, +so `StorePermalink`, `StoreFullText` and `StoreDigest` belong **before** this +plugin, where there is still one item per article to record. `PublishMarkdown` +heads the joined item with its title and writes no `Link` bullet. + +Nothing here is about AI. Joining a day's log lines, notifications or release +notes into one document is the same operation, and this plugin adds no prompt +of its own — what the joined text is for is decided by whatever the Recipe puts +next. + +```yaml + - module: FilterJoin + config: + title: Daily Digest +``` + +**The four AI filters.** The plugins that follow each send an item's +description to one AI service and put the answer back in its place. They are +four plugins rather than one with a `provider` setting, and that is the design +rather than an accident: the services differ in endpoint, authentication, +request body, answer shape, error format and available models; each of those +moves without asking the others; and a Recipe naming `FilterClaude` says on its +face where the text is being sent. +Changing service is changing that one line. + +**None of them is a summarizer.** The Recipe's `prompt` is the instruction and +the item's description is the text it applies to, so summarizing, translating, +extracting, reformatting and classifying are the same plugin with a different +prompt. There is no default prompt: a Recipe without one is refused with an +`ArgumentError` rather than being given a purpose it did not ask for. The two +are sent as separate fields — a system instruction and a user turn — so that +what an article says is text to be worked on, never an instruction to obey. + +What the four have in common: + +| Point | What it is | +| --- | --- | +| Required settings | `token`, `model` and `prompt`. A Recipe missing one is an `ArgumentError` before the first request. | +| `retry`, `interval` | Attempts after a failure, and seconds to wait between them. Both default to `0`. | +| What is retried | The network, a `429`, a `5xx`. | +| What is not | A refused request, an answer that is not JSON, an answer whose shape is not the one the service documents, and a setting that is missing or wrong. These raise and end the run, because the next attempt would fail the same way. | +| Input and output | The pipeline's feeds and items, in the same number and the same order. Only `description` is replaced; `title`, `link`, `date` and the rest are untouched. | +| An item with no description | Logged and passed over. Nothing is sent, and nothing is emptied. | +| A failure | Never leaves an empty description behind. A run that could not transform an item ends rather than publishing the article as a blank. | +| The credential | A Recipe setting, which makes the Recipe a secret file. It is never logged, never in an exception message, and never written into an item. TLS certificates are verified. | + +Each of them makes **one request per item**, which is what makes the order of a +Recipe worth thinking about: + +- `FilterJoin` → an AI filter: the articles become one text and the service is + asked about it **once**. This is the digest arrangement — one answer over + everything, which is not the same as a list of separate summaries. +- An AI filter → `FilterJoin`: each article is transformed **on its own**, and + the answers are joined afterwards. Use `FilterOne` or a store plugin ahead of + it on a large feed; each item is a billed request. + +#### FilterOpenAI — **Supported (external)** + +`filter/open_ai.rb`. Sends each item's description to the OpenAI API and +replaces it with the answer. It speaks the Responses API, +`https://api.openai.com/v1/responses`, which is the interface OpenAI recommends +for new integrations, and authenticates with the token as a bearer token. + +| Key | Type | Meaning | +| --- | --- | --- | +| `token` | string | OpenAI API key. Required. | +| `model` | string | Model name, as OpenAI names it. Required. | +| `prompt` | string | The instruction, sent as the request's `instructions`. Required. | +| `retry` | integer | Attempts after a failure. Default `0`. | +| `interval` | integer | Seconds between attempts. Default `0`. | + +The endpoint is not a setting: there is one, an operator has no version of this +plugin that talks to a different host, and a setting for it would be a way to +send the token somewhere else. The answer is read out of the typed `output` +array, from the `output_text` of the assistant's message. + +```yaml + - module: FilterOpenAI + config: + token: sk-... + model: gpt-5.6 + prompt: | + Summarize the following articles as one digest, in Japanese. + retry: 2 + interval: 2 +``` + +#### FilterClaude — **Supported (external)** + +`filter/claude.rb`. Sends each item's description to the Anthropic Messages +API, `https://api.anthropic.com/v1/messages`, and replaces it with the answer. +Anthropic authenticates with an `x-api-key` header rather than a bearer token, +requires an API version header, and requires a `max_tokens` — so this plugin +sends all three, and has one setting the others do not. + +| Key | Type | Meaning | +| --- | --- | --- | +| `token` | string | Anthropic API key, sent as `x-api-key`. Required. | +| `model` | string | Model name, as Anthropic names it. Required. | +| `prompt` | string | The instruction, sent as the request's `system`. Required. | +| `max_tokens` | integer | The longest answer to allow, which this API requires. Default `4096`. | +| `retry` | integer | Attempts after a failure. Default `0`. | +| `interval` | integer | Seconds between attempts. Default `0`. | + +The `anthropic-version` header is a constant, not a setting: it is the version +of the HTTP interface rather than of a model, and changing it is a change to +this plugin. The answer is the `text` of the content blocks the API returns; +blocks of other kinds are passed over. + +```yaml + - module: FilterClaude + config: + token: sk-ant-... + model: claude-opus-5 + prompt: | + Summarize the following articles as one digest, in Japanese. + max_tokens: 2048 + retry: 2 + interval: 2 +``` + +#### FilterGemini — **Supported (external)** + +`filter/gemini.rb`. Sends each item's description to the Google Gemini API and +replaces it with the answer. Gemini names the model in the URL rather than in +the body, so the endpoint is +`https://generativelanguage.googleapis.com/v1beta/models/:generateContent`, +built from the Recipe's `model`. The API key goes in an `x-goog-api-key` +header, which is how Google documents it and what keeps a credential out of a +URL and out of anything that logs one. + +| Key | Type | Meaning | +| --- | --- | --- | +| `token` | string | Gemini API key, sent as `x-goog-api-key`. Required. | +| `model` | string | Model name, bare — `gemini-3.5-flash`, not `models/gemini-3.5-flash`. Required. | +| `prompt` | string | The instruction, sent as `system_instruction`. Required. | +| `retry` | integer | Attempts after a failure. Default `0`. | +| `interval` | integer | Seconds between attempts. Default `0`. | + +The request is built of `contents` and `parts` as this API defines them, and is +not bent into another service's shape. The answer is the text of the first +candidate's parts; an answer carrying no candidate — which is what a request +stopped by a safety filter looks like — is an error rather than an empty +description. + +```yaml + - module: FilterGemini + config: + token: AIza... + model: gemini-3.5-flash + prompt: | + Summarize the following articles as one digest, in Japanese. + retry: 2 + interval: 2 +``` + +#### FilterSakuraAI — **Supported (external)** + +`filter/sakura_ai.rb`. Sends each item's description to the Sakura AI Engine, +`https://api.ai.sakura.ad.jp/v1/chat/completions`, and replaces it with the +answer. The token is a bearer token, and the request is the chat completions +form: the prompt as a `system` message, the description as a `user` message. + +| Key | Type | Meaning | +| --- | --- | --- | +| `token` | string | Sakura AI Engine token. Required. | +| `model` | string | Model name, as the service's control panel lists it. Required. | +| `prompt` | string | The instruction, sent as the `system` message. Required. | +| `retry` | integer | Attempts after a failure. Default `0`. | +| `interval` | integer | Seconds between attempts. Default `0`. | + +**This interface is OpenAI-compatible, and this is still its own plugin.** It is +a different service: a different endpoint, a different account, a different set +of models, its own limits and its own errors, any of which may move without +OpenAI moving. Folding it into `FilterOpenAI` behind a setting would trade a +Recipe that says where the text goes for a Recipe that does not. + +```yaml + - module: FilterSakuraAI + config: + token: ... + model: gpt-oss-120b + prompt: | + 以下の記事群について、個別記事の要約を羅列するのではなく、 + 全体を一つのダイジェストとして日本語で要約してください。 + retry: 2 + interval: 2 +``` + +A digest, end to end: find the articles, drop the ones already seen, fetch +their bodies, strip the markup, join them, ask once, write the answer out. + +```yaml +plugins: + - module: CustomFeedWeb + config: + sites: + - url: https://example.com/news/ + + - module: StorePermalink + config: + db: digest.db + + - module: FilterFullFeed + config: + siteinfo: items_all.json + + - module: FilterSanitize + + - module: FilterJoin + config: + title: Daily Digest + + - module: FilterSakuraAI + config: + token: ... + model: gpt-oss-120b + prompt: | + 以下の記事群について、個別記事の要約を羅列するのではなく、 + 全体を一つのダイジェストとして日本語で要約してください。 + retry: 2 + interval: 2 + + - module: PublishMarkdown + config: + file: ~/.automatic/markdown/digest.md + mode: append +``` + +Changing service is changing the one entry: `FilterSakuraAI` for +`FilterOpenAI`, `FilterClaude` or `FilterGemini`, with that plugin's own +settings. Nothing before or after it changes. + ### 6.4 Store Persist, and drop what has already been seen. A store plugin is what makes a @@ -1563,11 +1836,11 @@ is a claim that the plugin works. | Status | Count | Plugins | | --- | --- | --- | -| Supported | 25 | `SubscriptionFeed`, `SubscriptionLink`, `SubscriptionXml`, `SubscriptionText`, `CustomFeedWeb`, `FilterIgnore`, `FilterAccept`, `FilterSort`, `FilterOne`, `FilterRand`, `FilterClear`, `FilterImage`, `FilterImageSource`, `FilterAbsoluteURI`, `FilterSanitize`, `FilterTumblrResize`, `FilterDescriptionLink`, `FilterGithubFeed`, `StorePermalink`, `StoreFullText`, `StoreDigest`, `StoreFile`, `PublishMarkdown`, `PublishConsole`, `PublishConsoleLink` | -| Supported (external) | 10 | `SubscriptionTumblr`, `CustomFeedSVNLog`, `FilterFullFeed`, `ProvideFluentd`, `NotifyIkachan`, `PublishEject`, `PublishMemcached`, `PublishFluentd`, `PublishInstapaper`, `PublishAmazonS3` | +| Supported | 26 | `SubscriptionFeed`, `SubscriptionLink`, `SubscriptionXml`, `SubscriptionText`, `CustomFeedWeb`, `FilterIgnore`, `FilterAccept`, `FilterSort`, `FilterOne`, `FilterRand`, `FilterClear`, `FilterImage`, `FilterImageSource`, `FilterAbsoluteURI`, `FilterSanitize`, `FilterTumblrResize`, `FilterDescriptionLink`, `FilterGithubFeed`, `FilterJoin`, `StorePermalink`, `StoreFullText`, `StoreDigest`, `StoreFile`, `PublishMarkdown`, `PublishConsole`, `PublishConsoleLink` | +| Supported (external) | 14 | `SubscriptionTumblr`, `CustomFeedSVNLog`, `FilterFullFeed`, `FilterOpenAI`, `FilterClaude`, `FilterGemini`, `FilterSakuraAI`, `ProvideFluentd`, `NotifyIkachan`, `PublishEject`, `PublishMemcached`, `PublishFluentd`, `PublishInstapaper`, `PublishAmazonS3` | | Needs rework | 1 | `PublishHatenaBookmark` | -Thirty-six plugins. Every one of them either runs, or names the one thing it +Forty-one plugins. Every one of them either runs, or names the one thing it needs from the operator; the single exception says what is wrong with it and what fixing it would take. diff --git a/doc/VERSIONS b/doc/VERSIONS index 91d439d..5eadeab 100644 --- a/doc/VERSIONS +++ b/doc/VERSIONS @@ -13,6 +13,8 @@ v26.08 (Release Date: TBD) - Remove the plugins for Twitter, Pocket, HipChat, Google Calendar, livedoor Weather, So-net G-Guide and Chan-Toru, and Google News link rewriting; their services or APIs no longer exist, and a Recipe naming one now fails at load. - Add the plugin CustomFeedWeb, which builds one feed per HTML index page from the article links it lists, selected with CSS selectors and filtered by host, pattern and count. - Add the plugin StoreDigest, which drops items by content rather than by URL, on a SHA-256 digest of the item fields a Recipe names. +- Add the plugin FilterJoin, which joins the whole pipeline into one item carrying each item's title, link and body, and invents no permalink for it. +- Add the plugins FilterOpenAI, FilterClaude, FilterGemini and FilterSakuraAI, one per AI service rather than one selectable by setting, each replacing an item's description with what that service answers to the Recipe's prompt under a token the Recipe holds. - Rebuild the test and CI strategy for current RSpec and Ruby, with deterministic isolation from user data and external services. - Add Markdown as the primary service-independent publication format, with a documented and tested first-run workflow. - Rebuild the maintained documentation around current usage, architecture, policy, plugins and deployment, and remove superseded historical documents. diff --git a/plugins/filter/claude.rb b/plugins/filter/claude.rb new file mode 100644 index 0000000..43ca976 --- /dev/null +++ b/plugins/filter/claude.rb @@ -0,0 +1,217 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::Claude +# Description:: Replace each item's description with what the Anthropic Claude API answers. +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. +# +# One transformation: the item's description goes to the Anthropic Messages API +# under the Recipe's prompt, and the answer becomes the item's description. +# What that transformation is -- a summary, a translation, an extraction, a +# classification -- is the prompt's business, not this plugin's. +# +# This plugin knows Anthropic and nothing else. Its authentication is an +# `x-api-key` header rather than a bearer token, it requires an API version +# header and a `max_tokens`, and its answer is a list of content blocks. None +# of that is bent into another service's shape, and no other service's request +# is built here. +# +# @see https://docs.anthropic.com/en/api/messages + +module Automatic::Plugin + class FilterClaude + require 'json' + require 'net/http' + require 'openssl' + require 'uri' + + ENDPOINT = URI('https://api.anthropic.com/v1/messages') + + # The API version header the Messages API requires on every request. It is + # the version of the HTTP interface, not of a model, which is why it is a + # constant here and the model is a setting. + API_VERSION = '2023-06-01'.freeze + + # `max_tokens` is required by this API and by no other one this repository + # speaks to, so it is a setting of this plugin alone. The default is a + # length a digest or a translation fits in; a Recipe that wants a longer + # answer says so. + DEFAULT_MAX_TOKENS = 4096 + + OPEN_TIMEOUT = 10 + + # Generous, and bounded. A model given several articles thinks for a while; + # an unattended run that waits forever is the failure this exists against. + READ_TIMEOUT = 300 + + # A failure that another attempt will not get past: a setting that is + # wrong, a request the service refuses, an answer this plugin cannot read. + class Error < StandardError; end + + # A failure that another attempt may get past: the network, a rate limit, a + # server error. + class TemporaryError < StandardError; end + + def initialize(config, pipeline = []) + @config = config || {} + @pipeline = pipeline + end + + # Replaces each item's description with the answer. Nothing else about an + # item is touched, and the feeds and their items arrive and leave in the + # same order and number. + def run + validate_settings + + @pipeline.each { |feeds| + next if feeds.nil? + + feeds.items.each { |item| transform(item) } + } + @pipeline + end + + private + + # Checked before the first request, because a Recipe this plugin cannot + # carry out is the operator's mistake and will be the same mistake on every + # item. The token is never named in a message. + def validate_settings + raise ArgumentError, 'FilterClaude needs a token' if token.empty? + raise ArgumentError, 'FilterClaude needs a model' if model.empty? + raise ArgumentError, 'FilterClaude needs a prompt' if prompt.empty? + raise ArgumentError, 'FilterClaude needs a positive max_tokens' unless max_tokens.positive? + end + + def token + @config['token'].to_s + end + + def model + @config['model'].to_s.strip + end + + def prompt + @config['prompt'].to_s.strip + end + + def max_tokens + given = @config['max_tokens'] + given.nil? ? DEFAULT_MAX_TOKENS : given.to_i + end + + def transform(item) + text = item.description.to_s + if text.strip.empty? + Automatic::Log.puts('warn', "FilterClaude: nothing to send for #{item.link}") + return + end + + Automatic::Log.puts('info', "FilterClaude: asking #{model} about #{item.link}") + item.description = answer(text) + end + + # The retry shape of doc/PLUGINS.md section 3.6, applied only to what + # retrying can help. A missing setting, a refused request or an answer in a + # shape this plugin cannot read is raised at once: trying again would fail + # the same way, more slowly. + def answer(text) + retries = 0 + retry_max = @config['retry'].to_i + begin + message(text) + rescue TemporaryError => e + retries += 1 + Automatic::Log.puts('error', "ErrorCount: #{retries}, FilterClaude: #{e.message}") + if retries <= retry_max + sleep(@config['interval'].to_i) + retry + end + raise Error, "FilterClaude gave up after #{retries} attempts: #{e.message}" + end + end + + def message(text) + # The prompt is the system instruction and the description is the user + # turn it is applied to. They are separate fields, so that what an + # article says is never read as an instruction to this plugin or to the + # model. + body = { + 'model' => model, + 'max_tokens' => max_tokens, + 'system' => prompt, + 'messages' => [{ 'role' => 'user', 'content' => text }] + } + content(post(JSON.generate(body))) + end + + def post(body) + request = Net::HTTP::Post.new(ENDPOINT) + request['x-api-key'] = token + request['anthropic-version'] = API_VERSION + request['Content-Type'] = 'application/json' + request.body = body + + # TLS with the certificate verified, which is Net::HTTP's own default and + # is named here because it is not a thing to be turned off. + Net::HTTP.start(ENDPOINT.host, ENDPOINT.port, + use_ssl: true, + verify_mode: OpenSSL::SSL::VERIFY_PEER, + open_timeout: OPEN_TIMEOUT, + read_timeout: READ_TIMEOUT) { |http| http.request(request) } + rescue Timeout::Error, SystemCallError, SocketError, IOError, + OpenSSL::SSL::SSLError, Net::HTTPBadResponse => e + raise TemporaryError, "the request to Claude failed: #{e.message}" + end + + def content(response) + case response + when Net::HTTPSuccess + answer_text(parse(response.body)) + when Net::HTTPTooManyRequests, Net::HTTPServerError + raise TemporaryError, "Claude answered #{response.code}: #{reason(response)}" + else + raise Error, "Claude answered #{response.code}: #{reason(response)}" + end + end + + def parse(body) + JSON.parse(body.to_s) + rescue JSON::ParserError => e + raise Error, "Claude answered with something that is not JSON: #{e.message}" + end + + # The text of the answer, out of the content blocks the Messages API + # returns. A block of another type -- this API has several -- is not text + # and is passed over. An answer this plugin cannot find is an error and not + # an empty description: a Recipe that published the empty string here would + # have thrown the article away and reported success. + def answer_text(body) + blocks = body['content'] + raise Error, 'Claude answered without a content array' unless blocks.is_a?(Array) + + text = blocks.select { |block| block.is_a?(Hash) && block['type'] == 'text' }. + map { |block| block['text'].to_s }.join.strip + raise Error, 'Claude answered with no text content' if text.empty? + + text + end + + # The service's own explanation where it gave one, the status line + # otherwise. Neither carries the token, and the settings are never logged + # or raised wholesale. + def reason(response) + body = JSON.parse(response.body.to_s) + error = body['error'] + return error['message'].to_s if error.is_a?(Hash) && !error['message'].to_s.empty? + + response.message.to_s + rescue JSON::ParserError + response.message.to_s + end + end +end diff --git a/plugins/filter/gemini.rb b/plugins/filter/gemini.rb new file mode 100644 index 0000000..f0be5dd --- /dev/null +++ b/plugins/filter/gemini.rb @@ -0,0 +1,216 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::Gemini +# Description:: Replace each item's description with what the Google Gemini API answers. +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. +# +# One transformation: the item's description goes to the Gemini API under the +# Recipe's prompt, and the answer becomes the item's description. What that +# transformation is -- a summary, a translation, an extraction, a +# classification -- is the prompt's business, not this plugin's. +# +# This plugin knows Gemini and nothing else. Its model is named in the URL +# rather than in the body, its API key is a header of its own, and its request +# and answer are built of `contents` and `parts`. None of that is bent into +# another service's shape, and no other service's request is built here. +# +# It speaks `generateContent`, which Google states remains fully supported and +# is the single-turn text interface: a request that carries an instruction and +# a text and answers with a text, which is exactly what this plugin does. +# @see https://ai.google.dev/api/generate-content + +module Automatic::Plugin + class FilterGemini + require 'json' + require 'net/http' + require 'openssl' + require 'uri' + + # The model is part of the path here, unlike every other service in this + # directory, so the endpoint is built per Recipe rather than being one + # constant. + ENDPOINT_FORMAT = 'https://generativelanguage.googleapis.com/v1beta/models/%s:generateContent'.freeze + + OPEN_TIMEOUT = 10 + + # Generous, and bounded. A model given several articles thinks for a while; + # an unattended run that waits forever is the failure this exists against. + READ_TIMEOUT = 300 + + # A failure that another attempt will not get past: a setting that is + # wrong, a request the service refuses, an answer this plugin cannot read. + class Error < StandardError; end + + # A failure that another attempt may get past: the network, a rate limit, a + # server error. + class TemporaryError < StandardError; end + + def initialize(config, pipeline = []) + @config = config || {} + @pipeline = pipeline + end + + # Replaces each item's description with the answer. Nothing else about an + # item is touched, and the feeds and their items arrive and leave in the + # same order and number. + def run + validate_settings + + @pipeline.each { |feeds| + next if feeds.nil? + + feeds.items.each { |item| transform(item) } + } + @pipeline + end + + private + + # Checked before the first request, because a Recipe this plugin cannot + # carry out is the operator's mistake and will be the same mistake on every + # item. The token is never named in a message. + def validate_settings + raise ArgumentError, 'FilterGemini needs a token' if token.empty? + raise ArgumentError, 'FilterGemini needs a model' if model.empty? + raise ArgumentError, 'FilterGemini needs a prompt' if prompt.empty? + + endpoint + end + + def token + @config['token'].to_s + end + + def model + @config['model'].to_s.strip + end + + def prompt + @config['prompt'].to_s.strip + end + + # A model name that cannot go in a URL is the Recipe's mistake, and is + # reported as one rather than as a failed request. + def endpoint + @endpoint ||= URI(format(ENDPOINT_FORMAT, model)) + rescue URI::InvalidURIError + raise ArgumentError, "FilterGemini cannot build a request for the model #{model}" + end + + def transform(item) + text = item.description.to_s + if text.strip.empty? + Automatic::Log.puts('warn', "FilterGemini: nothing to send for #{item.link}") + return + end + + Automatic::Log.puts('info', "FilterGemini: asking #{model} about #{item.link}") + item.description = answer(text) + end + + # The retry shape of doc/PLUGINS.md section 3.6, applied only to what + # retrying can help. A missing setting, a refused request or an answer in a + # shape this plugin cannot read is raised at once: trying again would fail + # the same way, more slowly. + def answer(text) + retries = 0 + retry_max = @config['retry'].to_i + begin + generated(text) + rescue TemporaryError => e + retries += 1 + Automatic::Log.puts('error', "ErrorCount: #{retries}, FilterGemini: #{e.message}") + if retries <= retry_max + sleep(@config['interval'].to_i) + retry + end + raise Error, "FilterGemini gave up after #{retries} attempts: #{e.message}" + end + end + + def generated(text) + # The prompt is the system instruction and the description is the content + # it is applied to. They are separate fields, so that what an article + # says is never read as an instruction to this plugin or to the model. + body = { + 'system_instruction' => { 'parts' => [{ 'text' => prompt }] }, + 'contents' => [{ 'role' => 'user', 'parts' => [{ 'text' => text }] }] + } + content(post(JSON.generate(body))) + end + + def post(body) + request = Net::HTTP::Post.new(endpoint) + # The key goes in a header rather than in the query string, which is the + # way Google documents and the way that keeps a credential out of a URL. + request['x-goog-api-key'] = token + request['Content-Type'] = 'application/json' + request.body = body + + # TLS with the certificate verified, which is Net::HTTP's own default and + # is named here because it is not a thing to be turned off. + Net::HTTP.start(endpoint.host, endpoint.port, + use_ssl: true, + verify_mode: OpenSSL::SSL::VERIFY_PEER, + open_timeout: OPEN_TIMEOUT, + read_timeout: READ_TIMEOUT) { |http| http.request(request) } + rescue Timeout::Error, SystemCallError, SocketError, IOError, + OpenSSL::SSL::SSLError, Net::HTTPBadResponse => e + raise TemporaryError, "the request to Gemini failed: #{e.message}" + end + + def content(response) + case response + when Net::HTTPSuccess + answer_text(parse(response.body)) + when Net::HTTPTooManyRequests, Net::HTTPServerError + raise TemporaryError, "Gemini answered #{response.code}: #{reason(response)}" + else + raise Error, "Gemini answered #{response.code}: #{reason(response)}" + end + end + + def parse(body) + JSON.parse(body.to_s) + rescue JSON::ParserError => e + raise Error, "Gemini answered with something that is not JSON: #{e.message}" + end + + # The text of the first candidate. A response with no candidate is what a + # request stopped by a safety filter looks like, and it is an error rather + # than an empty description: a Recipe that published the empty string here + # would have thrown the article away and reported success. + def answer_text(body) + candidates = body['candidates'] + unless candidates.is_a?(Array) && candidates.first.is_a?(Hash) + raise Error, 'Gemini answered without a candidate' + end + + parts = candidates.first.dig('content', 'parts') + raise Error, 'Gemini answered without content parts' unless parts.is_a?(Array) + + text = parts.select { |part| part.is_a?(Hash) }.map { |part| part['text'].to_s }.join.strip + raise Error, 'Gemini answered with no text' if text.empty? + + text + end + + # The service's own explanation where it gave one, the status line + # otherwise. Neither carries the token, and the settings are never logged + # or raised wholesale. + def reason(response) + body = JSON.parse(response.body.to_s) + error = body['error'] + return error['message'].to_s if error.is_a?(Hash) && !error['message'].to_s.empty? + + response.message.to_s + rescue JSON::ParserError + response.message.to_s + end + end +end diff --git a/plugins/filter/join.rb b/plugins/filter/join.rb new file mode 100644 index 0000000..739a0a9 --- /dev/null +++ b/plugins/filter/join.rb @@ -0,0 +1,107 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::Join +# Description:: Join every item in the pipeline into one item. +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. +# +# Many items in, one item out. That is the whole of it: no summarizing, no +# fetching, no knowledge of what reads the result. What the joined description +# is for is decided by whatever the Recipe puts next -- an AI filter, a publish +# plugin, or nothing at all. +# +# The whole pipeline becomes one item rather than one item per feed, because +# the point of joining is to have a single text; a Recipe that wants one item +# per feed still has the feeds separate before this plugin runs. + +module Automatic::Plugin + class FilterJoin + require 'rss' + + # A title an operator has not set. Short and predictable, in the manner of + # PublishMarkdown's `(untitled)`; a Recipe that publishes this item names + # it in `title`. + DEFAULT_TITLE = 'Joined items'.freeze + + # Where one item ends and the next begins, for whatever reads the joined + # text. Plain lines rather than markup: the descriptions being joined may + # be HTML or text, and a delimiter that survives both is one that neither + # can be mistaken for. + HEADING = 'ARTICLE'.freeze + + def initialize(config, pipeline = []) + @config = config || {} + @pipeline = pipeline + end + + # Returns one feed holding one item, or an empty pipeline when there was + # nothing to join. An item saying nothing is worse than no item: the + # plugins after this one would publish it. + def run + items = collect + if items.empty? + Automatic::Log.puts('warn', 'FilterJoin: no items to join') + return [] + end + + Automatic::Log.puts('info', "FilterJoin: joining #{items.size} items into one") + [feed(title, description(items))] + end + + private + + def collect + @pipeline.each_with_object([]) { |feeds, items| + next if feeds.nil? + + items.concat(feeds.items) + } + end + + def title + given = @config['title'].to_s + given.empty? ? DEFAULT_TITLE : given + end + + def description(items) + items.each_with_index.map { |item, index| section(index + 1, item) }.join("\n\n") + end + + def section(number, item) + ["#{HEADING} #{number}", + "Title: #{value(item, :title)}", + "URL: #{value(item, :link)}", + '', + value(item, :description)].join("\n") + end + + # A field an item does not carry is empty rather than absent, so that every + # section has the same shape whatever the feed it came from left out. + def value(item, name) + return '' unless item.respond_to?(name) + + item.public_send(name).to_s.strip + end + + # Built here rather than through FeedMaker.create_pipeline, which drops an + # item whose link is nil -- and this item's link is nil deliberately. It is + # several articles at once, so there is no page it points at, and putting + # the first article's URL there would name a source for text that is not + # only from it. + def feed(item_title, item_description) + RSS::Maker.make('2.0') { |maker| + maker.channel.title = 'Automatic Ruby' + maker.channel.description = 'Automatic::Plugin::FilterJoin' + maker.channel.link = 'https://github.com/id774/automaticruby' + item = maker.items.new_item + item.title = item_title + item.description = item_description + item.date = Time.now + } + end + end +end diff --git a/plugins/filter/open_ai.rb b/plugins/filter/open_ai.rb new file mode 100644 index 0000000..d9bbdb8 --- /dev/null +++ b/plugins/filter/open_ai.rb @@ -0,0 +1,198 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::OpenAI +# Description:: Replace each item's description with what the OpenAI API answers. +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. +# +# One transformation: the item's description goes to the OpenAI API under the +# Recipe's prompt, and the answer becomes the item's description. What that +# transformation is -- a summary, a translation, an extraction, a +# classification -- is the prompt's business, not this plugin's. +# +# This plugin knows OpenAI and nothing else. Anthropic, Google and Sakura have +# their own plugins, because their endpoints, authentication, request bodies, +# responses and errors are their own and will keep diverging. A Recipe changes +# service by naming a different plugin. +# +# It speaks the Responses API, which is the interface OpenAI recommends for new +# integrations. @see https://platform.openai.com/docs/api-reference/responses + +module Automatic::Plugin + class FilterOpenAI + require 'json' + require 'net/http' + require 'openssl' + require 'uri' + + ENDPOINT = URI('https://api.openai.com/v1/responses') + + OPEN_TIMEOUT = 10 + + # Generous, and bounded. A model given several articles thinks for a while; + # an unattended run that waits forever is the failure this exists against. + READ_TIMEOUT = 300 + + # A failure that another attempt will not get past: a setting that is + # wrong, a request the service refuses, an answer this plugin cannot read. + class Error < StandardError; end + + # A failure that another attempt may get past: the network, a rate limit, a + # server error. + class TemporaryError < StandardError; end + + def initialize(config, pipeline = []) + @config = config || {} + @pipeline = pipeline + end + + # Replaces each item's description with the answer. Nothing else about an + # item is touched, and the feeds and their items arrive and leave in the + # same order and number. + def run + validate_settings + + @pipeline.each { |feeds| + next if feeds.nil? + + feeds.items.each { |item| transform(item) } + } + @pipeline + end + + private + + # Checked before the first request, because a Recipe this plugin cannot + # carry out is the operator's mistake and will be the same mistake on every + # item. The token is never named in a message. + def validate_settings + raise ArgumentError, 'FilterOpenAI needs a token' if token.empty? + raise ArgumentError, 'FilterOpenAI needs a model' if model.empty? + raise ArgumentError, 'FilterOpenAI needs a prompt' if prompt.empty? + end + + def token + @config['token'].to_s + end + + def model + @config['model'].to_s.strip + end + + def prompt + @config['prompt'].to_s.strip + end + + def transform(item) + text = item.description.to_s + if text.strip.empty? + Automatic::Log.puts('warn', "FilterOpenAI: nothing to send for #{item.link}") + return + end + + Automatic::Log.puts('info', "FilterOpenAI: asking #{model} about #{item.link}") + item.description = answer(text) + end + + # The retry shape of doc/PLUGINS.md section 3.6, applied only to what + # retrying can help. A missing setting, a refused request or an answer in a + # shape this plugin cannot read is raised at once: trying again would fail + # the same way, more slowly. + def answer(text) + retries = 0 + retry_max = @config['retry'].to_i + begin + completion(text) + rescue TemporaryError => e + retries += 1 + Automatic::Log.puts('error', "ErrorCount: #{retries}, FilterOpenAI: #{e.message}") + if retries <= retry_max + sleep(@config['interval'].to_i) + retry + end + raise Error, "FilterOpenAI gave up after #{retries} attempts: #{e.message}" + end + end + + def completion(text) + # The prompt is the instruction and the description is the text it is + # applied to. They are separate fields, so that what an article says is + # never read as an instruction to this plugin or to the model. + body = { + 'model' => model, + 'instructions' => prompt, + 'input' => text + } + content(post(JSON.generate(body))) + end + + def post(body) + request = Net::HTTP::Post.new(ENDPOINT) + request['Authorization'] = "Bearer #{token}" + request['Content-Type'] = 'application/json' + request.body = body + + # TLS with the certificate verified, which is Net::HTTP's own default and + # is named here because it is not a thing to be turned off. + Net::HTTP.start(ENDPOINT.host, ENDPOINT.port, + use_ssl: true, + verify_mode: OpenSSL::SSL::VERIFY_PEER, + open_timeout: OPEN_TIMEOUT, + read_timeout: READ_TIMEOUT) { |http| http.request(request) } + rescue Timeout::Error, SystemCallError, SocketError, IOError, + OpenSSL::SSL::SSLError, Net::HTTPBadResponse => e + raise TemporaryError, "the request to OpenAI failed: #{e.message}" + end + + def content(response) + case response + when Net::HTTPSuccess + answer_text(parse(response.body)) + when Net::HTTPTooManyRequests, Net::HTTPServerError + raise TemporaryError, "OpenAI answered #{response.code}: #{reason(response)}" + else + raise Error, "OpenAI answered #{response.code}: #{reason(response)}" + end + end + + def parse(body) + JSON.parse(body.to_s) + rescue JSON::ParserError => e + raise Error, "OpenAI answered with something that is not JSON: #{e.message}" + end + + # The text of the assistant's message, out of the typed output array the + # Responses API returns. An answer this plugin cannot find is an error and + # not an empty description: a Recipe that published the empty string here + # would have thrown the article away and reported success. + def answer_text(body) + output = body['output'] + raise Error, 'OpenAI answered without an output array' unless output.is_a?(Array) + + text = output.select { |item| item.is_a?(Hash) && item['type'] == 'message' }. + flat_map { |item| Array(item['content']) }. + select { |part| part.is_a?(Hash) && part['type'] == 'output_text' }. + map { |part| part['text'].to_s }.join.strip + raise Error, 'OpenAI answered with no output text' if text.empty? + + text + end + + # The service's own explanation where it gave one, the status line + # otherwise. Neither carries the token, and the settings are never logged + # or raised wholesale. + def reason(response) + body = JSON.parse(response.body.to_s) + error = body['error'] + return error['message'].to_s if error.is_a?(Hash) && !error['message'].to_s.empty? + + response.message.to_s + rescue JSON::ParserError + response.message.to_s + end + end +end diff --git a/plugins/filter/sakura_ai.rb b/plugins/filter/sakura_ai.rb new file mode 100644 index 0000000..3e2554d --- /dev/null +++ b/plugins/filter/sakura_ai.rb @@ -0,0 +1,205 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::SakuraAI +# Description:: Replace each item's description with what the Sakura AI Engine answers. +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. +# +# One transformation: the item's description goes to the Sakura AI Engine under +# the Recipe's prompt, and the answer becomes the item's description. What that +# transformation is -- a summary, a translation, an extraction, a +# classification -- is the prompt's business, not this plugin's. +# +# The Sakura AI Engine offers an OpenAI-compatible chat completions interface, +# and this plugin is still its own rather than a mode of FilterOpenAI. It is a +# different service: a different endpoint, a different account, a different set +# of models, its own limits and its own errors, and any of those may move +# without OpenAI moving. A Recipe naming FilterSakuraAI says which service the +# text is sent to, which a `provider:` setting would not. +# +# @see https://manual.sakura.ad.jp/cloud/ai-engine/ + +module Automatic::Plugin + class FilterSakuraAI + require 'json' + require 'net/http' + require 'openssl' + require 'uri' + + # The one endpoint the service publishes for this. It is not a setting: an + # operator has no version of this plugin that talks to a different host, + # and a setting for it would be a way to send the token somewhere else. + ENDPOINT = URI('https://api.ai.sakura.ad.jp/v1/chat/completions') + + OPEN_TIMEOUT = 10 + + # Generous, and bounded. A model given several articles thinks for a while; + # an unattended run that waits forever is the failure this exists against. + READ_TIMEOUT = 300 + + # A failure that another attempt will not get past: a setting that is + # wrong, a request the service refuses, an answer this plugin cannot read. + class Error < StandardError; end + + # A failure that another attempt may get past: the network, a rate limit, a + # server error. + class TemporaryError < StandardError; end + + def initialize(config, pipeline = []) + @config = config || {} + @pipeline = pipeline + end + + # Replaces each item's description with the answer. Nothing else about an + # item is touched, and the feeds and their items arrive and leave in the + # same order and number. + def run + validate_settings + + @pipeline.each { |feeds| + next if feeds.nil? + + feeds.items.each { |item| transform(item) } + } + @pipeline + end + + private + + # Checked before the first request, because a Recipe this plugin cannot + # carry out is the operator's mistake and will be the same mistake on every + # item. The token is never named in a message. + def validate_settings + raise ArgumentError, 'FilterSakuraAI needs a token' if token.empty? + raise ArgumentError, 'FilterSakuraAI needs a model' if model.empty? + raise ArgumentError, 'FilterSakuraAI needs a prompt' if prompt.empty? + end + + def token + @config['token'].to_s + end + + def model + @config['model'].to_s.strip + end + + def prompt + @config['prompt'].to_s.strip + end + + def transform(item) + text = item.description.to_s + if text.strip.empty? + Automatic::Log.puts('warn', "FilterSakuraAI: nothing to send for #{item.link}") + return + end + + Automatic::Log.puts('info', "FilterSakuraAI: asking #{model} about #{item.link}") + item.description = answer(text) + end + + # The retry shape of doc/PLUGINS.md section 3.6, applied only to what + # retrying can help. A missing setting, a refused request or an answer in a + # shape this plugin cannot read is raised at once: trying again would fail + # the same way, more slowly. + def answer(text) + retries = 0 + retry_max = @config['retry'].to_i + begin + completion(text) + rescue TemporaryError => e + retries += 1 + Automatic::Log.puts('error', "ErrorCount: #{retries}, FilterSakuraAI: #{e.message}") + if retries <= retry_max + sleep(@config['interval'].to_i) + retry + end + raise Error, "FilterSakuraAI gave up after #{retries} attempts: #{e.message}" + end + end + + def completion(text) + # The prompt is the system turn and the description is the user turn it + # is applied to. They are separate messages, so that what an article says + # is never read as an instruction to this plugin or to the model. + body = { + 'model' => model, + 'messages' => [ + { 'role' => 'system', 'content' => prompt }, + { 'role' => 'user', 'content' => text } + ] + } + content(post(JSON.generate(body))) + end + + def post(body) + request = Net::HTTP::Post.new(ENDPOINT) + request['Authorization'] = "Bearer #{token}" + request['Content-Type'] = 'application/json' + request.body = body + + # TLS with the certificate verified, which is Net::HTTP's own default and + # is named here because it is not a thing to be turned off. + Net::HTTP.start(ENDPOINT.host, ENDPOINT.port, + use_ssl: true, + verify_mode: OpenSSL::SSL::VERIFY_PEER, + open_timeout: OPEN_TIMEOUT, + read_timeout: READ_TIMEOUT) { |http| http.request(request) } + rescue Timeout::Error, SystemCallError, SocketError, IOError, + OpenSSL::SSL::SSLError, Net::HTTPBadResponse => e + raise TemporaryError, "the request to the Sakura AI Engine failed: #{e.message}" + end + + def content(response) + case response + when Net::HTTPSuccess + answer_text(parse(response.body)) + when Net::HTTPTooManyRequests, Net::HTTPServerError + raise TemporaryError, "the Sakura AI Engine answered #{response.code}: #{reason(response)}" + else + raise Error, "the Sakura AI Engine answered #{response.code}: #{reason(response)}" + end + end + + def parse(body) + JSON.parse(body.to_s) + rescue JSON::ParserError => e + raise Error, "the Sakura AI Engine answered with something that is not JSON: #{e.message}" + end + + # The first choice's message content. An answer this plugin cannot find is + # an error and not an empty description: a Recipe that published the empty + # string here would have thrown the article away and reported success. + def answer_text(body) + choices = body['choices'] + unless choices.is_a?(Array) && choices.first.is_a?(Hash) + raise Error, 'the Sakura AI Engine answered without a choice' + end + + message = choices.first['message'] + raise Error, 'the Sakura AI Engine answered without a message' unless message.is_a?(Hash) + + text = message['content'].to_s.strip + raise Error, 'the Sakura AI Engine answered with no content' if text.empty? + + text + end + + # The service's own explanation where it gave one, the status line + # otherwise. Neither carries the token, and the settings are never logged + # or raised wholesale. + def reason(response) + body = JSON.parse(response.body.to_s) + error = body['error'] + return error['message'].to_s if error.is_a?(Hash) && !error['message'].to_s.empty? + + response.message.to_s + rescue JSON::ParserError + response.message.to_s + end + end +end diff --git a/spec/plugins/filter/claude_spec.rb b/spec/plugins/filter/claude_spec.rb new file mode 100644 index 0000000..940eb64 --- /dev/null +++ b/spec/plugins/filter/claude_spec.rb @@ -0,0 +1,308 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::Claude +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. + +require File.expand_path(File.dirname(__FILE__) + '../../../spec_helper') + +require 'filter/claude' +require 'json' +require 'net/http' + +# FilterClaude needs an Anthropic API token; doc/PLUGINS.md section 6.3 +# classifies it as Supported (external). Everything up to the request is +# verified here -- what is posted where, with which authentication, and what +# each answer means -- and no example reaches the service. Only Anthropic is +# exercised: the other AI filters have their own specs, and none of them stands +# in for another. +module ClaudeSpec + module_function + + # A real response object, because the plugin decides what to do from the + # class Net::HTTP would have given it. + def response(klass, code, body) + response = klass.new('1.1', code, 'Status') + response.instance_variable_set(:@body, body) + response.instance_variable_set(:@read, true) + response + end + + def answered(text) + response(Net::HTTPOK, '200', JSON.generate( + 'type' => 'message', + 'role' => 'assistant', + 'content' => [{ 'type' => 'text', 'text' => text }], + 'stop_reason' => 'end_turn' + )) + end + + def refused(klass, code, message) + response(klass, code, JSON.generate( + 'type' => 'error', 'error' => { 'type' => 'invalid_request_error', 'message' => message } + )) + end +end + +describe Automatic::Plugin::FilterClaude do + let(:settings) { + { 'token' => 'test-token', 'model' => 'claude-test', 'prompt' => 'Summarize this.', + 'retry' => 1, 'interval' => 0 } + } + + let(:posted) { [] } + let(:connections) { [] } + + def one_item + AutomaticSpec.generate_pipeline { + feed { item 'https://example.com/a', 'A', 'the body of A' } + } + end + + # Stands in for the network: records the connection and the request, answers + # with what the example queued, and opens nothing. The default suite reaches + # no network (doc/POLICY.md Invariant 6). + def serve(*responses) + requests = posted + opened = connections + Net::HTTP.stub(:start) { |*args, &block| + opened << args + http = double('http') + http.stub(:request) { |request| + requests << request + responses.shift + } + block.call(http) + } + end + + def run(config, pipeline) + Automatic::Plugin::FilterClaude.new(config, pipeline).run + end + + describe 'the request it builds' do + before { + serve(ClaudeSpec.answered('a summary')) + run(settings, one_item) + } + + it 'posts to the Messages API over TLS, with the certificate verified' do + connections[0][0].should == 'api.anthropic.com' + connections[0][1].should == 443 + connections[0].last[:use_ssl].should == true + connections[0].last[:verify_mode].should == OpenSSL::SSL::VERIFY_PEER + posted[0].path.should == '/v1/messages' + end + + it 'sends the token in the header this API authenticates with' do + posted[0]['x-api-key'].should == 'test-token' + posted[0]['authorization'].should be_nil + posted[0]['content-type'].should == 'application/json' + end + + it 'sends the API version this API requires' do + posted[0]['anthropic-version'].should == Automatic::Plugin::FilterClaude::API_VERSION + end + + it 'sends the model, the prompt as the system turn and the description as the user turn' do + JSON.parse(posted[0].body).should == { + 'model' => 'claude-test', + 'max_tokens' => Automatic::Plugin::FilterClaude::DEFAULT_MAX_TOKENS, + 'system' => 'Summarize this.', + 'messages' => [{ 'role' => 'user', 'content' => 'the body of A' }] + } + end + end + + describe 'max_tokens' do + it 'takes the length the Recipe asks for' do + serve(ClaudeSpec.answered('a summary')) + run(settings.merge('max_tokens' => 256), one_item) + JSON.parse(posted[0].body)['max_tokens'].should == 256 + end + + it 'refuses a length that is not one' do + lambda { run(settings.merge('max_tokens' => 0), one_item) }. + should raise_error(ArgumentError, /max_tokens/) + end + end + + describe 'what it does to the pipeline' do + it 'replaces the description with the answer and leaves the rest alone' do + serve(ClaudeSpec.answered('a summary')) + returned = run(settings, one_item) + + returned.should have(1).feed + returned[0].items.should have(1).item + returned[0].items[0].description.should == 'a summary' + returned[0].items[0].title.should == 'A' + returned[0].items[0].link.should == 'https://example.com/a' + end + + it 'asks once for each item' do + serve(ClaudeSpec.answered('first summary'), ClaudeSpec.answered('second summary')) + returned = run(settings, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', 'the body of A' + item 'https://example.com/b', 'B', 'the body of B' + } + }) + + posted.should have(2).requests + JSON.parse(posted[0].body)['messages'][0]['content'].should == 'the body of A' + JSON.parse(posted[1].body)['messages'][0]['content'].should == 'the body of B' + returned[0].items.map(&:description).should == ['first summary', 'second summary'] + end + + it 'ignores a feed that is nil' do + serve(ClaudeSpec.answered('a summary')) + run(settings, [nil] + one_item).should have(2).feeds + posted.should have(1).request + end + + it 'sends nothing for an item with no description, and empties none' do + serve + returned = run(settings, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', '' + item 'https://example.com/b', 'B', ' ' + } + }) + + posted.should be_empty + returned[0].items.map(&:description).should == ['', ' '] + end + + it 'reads the text out of an answer that carries blocks of other kinds' do + serve(ClaudeSpec.response(Net::HTTPOK, '200', JSON.generate( + 'content' => [ + { 'type' => 'thinking', 'thinking' => '' }, + { 'type' => 'text', 'text' => 'a summary' } + ] + ))) + run(settings, one_item)[0].items[0].description.should == 'a summary' + end + end + + describe 'when the request fails' do + it 'retries a server error and carries on' do + serve(ClaudeSpec.refused(Net::HTTPInternalServerError, '500', 'server error'), + ClaudeSpec.answered('a summary')) + returned = run(settings, one_item) + + posted.should have(2).requests + returned[0].items[0].description.should == 'a summary' + end + + it 'retries a rate limit' do + serve(ClaudeSpec.refused(Net::HTTPTooManyRequests, '429', 'rate limited'), + ClaudeSpec.answered('a summary')) + run(settings, one_item)[0].items[0].description.should == 'a summary' + posted.should have(2).requests + end + + it 'gives up after the configured number of retries' do + serve(ClaudeSpec.refused(Net::HTTPInternalServerError, '500', 'server error'), + ClaudeSpec.refused(Net::HTTPInternalServerError, '500', 'server error')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterClaude::Error, /gave up/) + posted.should have(2).requests + end + + it 'does not retry a rejected request' do + serve(ClaudeSpec.refused(Net::HTTPUnauthorized, '401', 'invalid x-api-key')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterClaude::Error, /401/) + posted.should have(1).request + end + + it 'does not empty the description when the service fails' do + serve(ClaudeSpec.refused(Net::HTTPUnauthorized, '401', 'invalid x-api-key')) + pipeline = one_item + lambda { run(settings, pipeline) }.should raise_error(StandardError) + pipeline[0].items[0].description.should == 'the body of A' + end + end + + describe 'when the answer cannot be read' do + it 'raises on a body that is not JSON' do + serve(ClaudeSpec.response(Net::HTTPOK, '200', 'not json at all')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterClaude::Error, /not JSON/) + posted.should have(1).request + end + + it 'raises on a body without the content it expects' do + serve(ClaudeSpec.response(Net::HTTPOK, '200', JSON.generate('id' => 'msg_1'))) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterClaude::Error, /content array/) + end + + it 'raises rather than writing an empty description' do + serve(ClaudeSpec.response(Net::HTTPOK, '200', JSON.generate('content' => []))) + pipeline = one_item + lambda { run(settings, pipeline) }. + should raise_error(Automatic::Plugin::FilterClaude::Error, /no text content/) + pipeline[0].items[0].description.should == 'the body of A' + end + end + + describe 'the settings it requires' do + it 'refuses a Recipe with no token' do + lambda { run(settings.merge('token' => nil), one_item) }. + should raise_error(ArgumentError, /token/) + end + + it 'refuses a Recipe with no model' do + lambda { run(settings.merge('model' => ''), one_item) }. + should raise_error(ArgumentError, /model/) + end + + it 'refuses a Recipe with no prompt' do + lambda { run(settings.merge('prompt' => nil), one_item) }. + should raise_error(ArgumentError, /prompt/) + end + + it 'asks nothing before it has what it needs' do + serve + lambda { run(settings.merge('prompt' => nil), one_item) }.should raise_error(ArgumentError) + posted.should be_empty + end + end + + describe 'the credential' do + # doc/PLUGINS.md section 3.7: never logged, never in an exception message, + # never written into an item. + it 'reaches neither the log nor the error, on the path that fails' do + messages = [] + logger = double('logger') + %i[info warn error].each { |level| logger.stub(level) { |message| messages << message.to_s } } + original = Automatic::Log.logger + + serve(ClaudeSpec.refused(Net::HTTPUnauthorized, '401', 'invalid x-api-key')) + begin + Automatic::Log.logger = logger + Automatic::Log.level('info') + lambda { run(settings, one_item) }.should raise_error(StandardError) { |error| + error.message.should_not include('test-token') + } + ensure + Automatic::Log.level('none') + Automatic::Log.logger = original + end + + messages.should_not be_empty + messages.each { |message| message.should_not include('test-token') } + end + + it 'writes nothing of itself into the item' do + serve(ClaudeSpec.answered('a summary')) + returned = run(settings, one_item) + returned[0].items[0].description.should_not include('test-token') + end + end +end diff --git a/spec/plugins/filter/gemini_spec.rb b/spec/plugins/filter/gemini_spec.rb new file mode 100644 index 0000000..de28def --- /dev/null +++ b/spec/plugins/filter/gemini_spec.rb @@ -0,0 +1,291 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::Gemini +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. + +require File.expand_path(File.dirname(__FILE__) + '../../../spec_helper') + +require 'filter/gemini' +require 'json' +require 'net/http' + +# FilterGemini needs a Gemini API key; doc/PLUGINS.md section 6.3 classifies it +# as Supported (external). Everything up to the request is verified here -- +# what is posted where, with which authentication, and what each answer means +# -- and no example reaches the service. Only Gemini is exercised: the other AI +# filters have their own specs, and none of them stands in for another. +module GeminiSpec + module_function + + # A real response object, because the plugin decides what to do from the + # class Net::HTTP would have given it. + def response(klass, code, body) + response = klass.new('1.1', code, 'Status') + response.instance_variable_set(:@body, body) + response.instance_variable_set(:@read, true) + response + end + + def answered(text) + response(Net::HTTPOK, '200', JSON.generate( + 'candidates' => [ + { 'content' => { 'role' => 'model', 'parts' => [{ 'text' => text }] }, + 'finishReason' => 'STOP' } + ] + )) + end + + def refused(klass, code, message) + response(klass, code, JSON.generate( + 'error' => { 'code' => code.to_i, 'message' => message, 'status' => 'ERROR' } + )) + end +end + +describe Automatic::Plugin::FilterGemini do + let(:settings) { + { 'token' => 'test-token', 'model' => 'gemini-test', 'prompt' => 'Summarize this.', + 'retry' => 1, 'interval' => 0 } + } + + let(:posted) { [] } + let(:connections) { [] } + + def one_item + AutomaticSpec.generate_pipeline { + feed { item 'https://example.com/a', 'A', 'the body of A' } + } + end + + # Stands in for the network: records the connection and the request, answers + # with what the example queued, and opens nothing. The default suite reaches + # no network (doc/POLICY.md Invariant 6). + def serve(*responses) + requests = posted + opened = connections + Net::HTTP.stub(:start) { |*args, &block| + opened << args + http = double('http') + http.stub(:request) { |request| + requests << request + responses.shift + } + block.call(http) + } + end + + def run(config, pipeline) + Automatic::Plugin::FilterGemini.new(config, pipeline).run + end + + describe 'the request it builds' do + before { + serve(GeminiSpec.answered('a summary')) + run(settings, one_item) + } + + it 'posts to generateContent for the configured model, over verified TLS' do + connections[0][0].should == 'generativelanguage.googleapis.com' + connections[0][1].should == 443 + connections[0].last[:use_ssl].should == true + connections[0].last[:verify_mode].should == OpenSSL::SSL::VERIFY_PEER + posted[0].path.should == '/v1beta/models/gemini-test:generateContent' + end + + it 'sends the key in the header rather than in the URL' do + posted[0]['x-goog-api-key'].should == 'test-token' + posted[0]['content-type'].should == 'application/json' + posted[0].path.should_not include('test-token') + end + + it 'sends the prompt as the system instruction and the description as the content' do + JSON.parse(posted[0].body).should == { + 'system_instruction' => { 'parts' => [{ 'text' => 'Summarize this.' }] }, + 'contents' => [{ 'role' => 'user', 'parts' => [{ 'text' => 'the body of A' }] }] + } + end + end + + describe 'what it does to the pipeline' do + it 'replaces the description with the answer and leaves the rest alone' do + serve(GeminiSpec.answered('a summary')) + returned = run(settings, one_item) + + returned.should have(1).feed + returned[0].items.should have(1).item + returned[0].items[0].description.should == 'a summary' + returned[0].items[0].title.should == 'A' + returned[0].items[0].link.should == 'https://example.com/a' + end + + it 'asks once for each item' do + serve(GeminiSpec.answered('first summary'), GeminiSpec.answered('second summary')) + returned = run(settings, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', 'the body of A' + item 'https://example.com/b', 'B', 'the body of B' + } + }) + + posted.should have(2).requests + JSON.parse(posted[0].body)['contents'][0]['parts'][0]['text'].should == 'the body of A' + JSON.parse(posted[1].body)['contents'][0]['parts'][0]['text'].should == 'the body of B' + returned[0].items.map(&:description).should == ['first summary', 'second summary'] + end + + it 'ignores a feed that is nil' do + serve(GeminiSpec.answered('a summary')) + run(settings, [nil] + one_item).should have(2).feeds + posted.should have(1).request + end + + it 'sends nothing for an item with no description, and empties none' do + serve + returned = run(settings, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', '' + item 'https://example.com/b', 'B', ' ' + } + }) + + posted.should be_empty + returned[0].items.map(&:description).should == ['', ' '] + end + + it 'joins the parts of a candidate that answers in several' do + serve(GeminiSpec.response(Net::HTTPOK, '200', JSON.generate( + 'candidates' => [ + { 'content' => { 'parts' => [{ 'text' => 'a ' }, { 'text' => 'summary' }] } } + ] + ))) + run(settings, one_item)[0].items[0].description.should == 'a summary' + end + end + + describe 'when the request fails' do + it 'retries a server error and carries on' do + serve(GeminiSpec.refused(Net::HTTPInternalServerError, '500', 'internal error'), + GeminiSpec.answered('a summary')) + returned = run(settings, one_item) + + posted.should have(2).requests + returned[0].items[0].description.should == 'a summary' + end + + it 'retries a rate limit' do + serve(GeminiSpec.refused(Net::HTTPTooManyRequests, '429', 'resource exhausted'), + GeminiSpec.answered('a summary')) + run(settings, one_item)[0].items[0].description.should == 'a summary' + posted.should have(2).requests + end + + it 'gives up after the configured number of retries' do + serve(GeminiSpec.refused(Net::HTTPInternalServerError, '500', 'internal error'), + GeminiSpec.refused(Net::HTTPInternalServerError, '500', 'internal error')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterGemini::Error, /gave up/) + posted.should have(2).requests + end + + it 'does not retry a rejected request' do + serve(GeminiSpec.refused(Net::HTTPBadRequest, '400', 'API key not valid')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterGemini::Error, /400/) + posted.should have(1).request + end + + it 'does not empty the description when the service fails' do + serve(GeminiSpec.refused(Net::HTTPBadRequest, '400', 'API key not valid')) + pipeline = one_item + lambda { run(settings, pipeline) }.should raise_error(StandardError) + pipeline[0].items[0].description.should == 'the body of A' + end + end + + describe 'when the answer cannot be read' do + it 'raises on a body that is not JSON' do + serve(GeminiSpec.response(Net::HTTPOK, '200', 'not json at all')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterGemini::Error, /not JSON/) + posted.should have(1).request + end + + it 'raises on an answer with no candidate, which is what a blocked request looks like' do + serve(GeminiSpec.response(Net::HTTPOK, '200', JSON.generate( + 'promptFeedback' => { 'blockReason' => 'SAFETY' } + ))) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterGemini::Error, /without a candidate/) + end + + it 'raises rather than writing an empty description' do + serve(GeminiSpec.response(Net::HTTPOK, '200', JSON.generate( + 'candidates' => [{ 'content' => { 'parts' => [] } }] + ))) + pipeline = one_item + lambda { run(settings, pipeline) }. + should raise_error(Automatic::Plugin::FilterGemini::Error, /no text/) + pipeline[0].items[0].description.should == 'the body of A' + end + end + + describe 'the settings it requires' do + it 'refuses a Recipe with no token' do + lambda { run(settings.merge('token' => nil), one_item) }. + should raise_error(ArgumentError, /token/) + end + + it 'refuses a Recipe with no model' do + lambda { run(settings.merge('model' => ''), one_item) }. + should raise_error(ArgumentError, /model/) + end + + it 'refuses a Recipe with no prompt' do + lambda { run(settings.merge('prompt' => nil), one_item) }. + should raise_error(ArgumentError, /prompt/) + end + + it 'asks nothing before it has what it needs' do + serve + lambda { run(settings.merge('prompt' => nil), one_item) }.should raise_error(ArgumentError) + posted.should be_empty + end + end + + describe 'the credential' do + # doc/PLUGINS.md section 3.7: never logged, never in an exception message, + # never written into an item. + it 'reaches neither the log nor the error, on the path that fails' do + messages = [] + logger = double('logger') + %i[info warn error].each { |level| logger.stub(level) { |message| messages << message.to_s } } + original = Automatic::Log.logger + + serve(GeminiSpec.refused(Net::HTTPBadRequest, '400', 'API key not valid')) + begin + Automatic::Log.logger = logger + Automatic::Log.level('info') + lambda { run(settings, one_item) }.should raise_error(StandardError) { |error| + error.message.should_not include('test-token') + } + ensure + Automatic::Log.level('none') + Automatic::Log.logger = original + end + + messages.should_not be_empty + messages.each { |message| message.should_not include('test-token') } + end + + it 'writes nothing of itself into the item' do + serve(GeminiSpec.answered('a summary')) + returned = run(settings, one_item) + returned[0].items[0].description.should_not include('test-token') + end + end +end diff --git a/spec/plugins/filter/join_spec.rb b/spec/plugins/filter/join_spec.rb new file mode 100644 index 0000000..2b331a8 --- /dev/null +++ b/spec/plugins/filter/join_spec.rb @@ -0,0 +1,155 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::Join +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. + +require File.expand_path(File.dirname(__FILE__) + '../../../spec_helper') + +require 'filter/join' + +describe Automatic::Plugin::FilterJoin do + def join(config, pipeline) + Automatic::Plugin::FilterJoin.new(config, pipeline).run + end + + describe 'what it returns' do + it 'joins one feed of one item into one feed of one item' do + returned = join({}, AutomaticSpec.generate_pipeline { + feed { item 'https://example.com/a', 'A', 'the body of A' } + }) + + returned.should have(1).feed + returned[0].items.should have(1).item + returned[0].items[0].description.should include('the body of A') + end + + it 'joins several items of one feed into one item' do + returned = join({}, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', 'the body of A' + item 'https://example.com/b', 'B', 'the body of B' + item 'https://example.com/c', 'C', 'the body of C' + } + }) + + returned.should have(1).feed + returned[0].items.should have(1).item + description = returned[0].items[0].description + %w[A B C].each { |title| description.should include("Title: #{title}") } + end + + it 'joins the whole pipeline, not each feed' do + returned = join({}, AutomaticSpec.generate_pipeline { + feed { item 'https://example.com/a', 'A', 'the body of A' } + feed { + item 'https://example.com/b', 'B', 'the body of B' + item 'https://example.com/c', 'C', 'the body of C' + } + }) + + returned.should have(1).feed + returned[0].items.should have(1).item + returned[0].items[0].description.should include('the body of C') + end + + it 'ignores a feed that is nil' do + pipeline = AutomaticSpec.generate_pipeline { + feed { item 'https://example.com/a', 'A', 'the body of A' } + } + returned = join({}, [nil] + pipeline + [nil]) + + returned.should have(1).feed + returned[0].items.should have(1).item + returned[0].items[0].description.should include('the body of A') + end + end + + describe 'when there is nothing to join' do + it 'returns an empty pipeline for an empty one' do + join({}, []).should == [] + end + + it 'returns an empty pipeline when every feed is nil' do + join({}, [nil, nil]).should == [] + end + + it 'makes no item out of feeds that carry none' do + join({}, AutomaticSpec.generate_pipeline { feed {} }).should == [] + end + end + + describe 'the joined description' do + subject { + join({}, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'Ruby 4.1 released', 'the body of A' + item 'https://example.com/b', 'PostgreSQL 19 released', 'the body of B' + } + })[0].items[0].description + } + + it 'numbers each item so that what reads it can tell them apart' do + subject.should include('ARTICLE 1') + subject.should include('ARTICLE 2') + end + + it 'carries the title, the link and the body of each item' do + subject.should include('Title: Ruby 4.1 released') + subject.should include('URL: https://example.com/a') + subject.should include('the body of A') + end + + it 'keeps the items in the order they arrived' do + subject.index('Ruby 4.1 released').should < subject.index('PostgreSQL 19 released') + end + + it 'adds no prompt of its own' do + subject.should_not match(/summar|要約/i) + end + + it 'treats a missing title, link or description as empty' do + description = join({}, AutomaticSpec.generate_pipeline { + feed { item nil, '', '' } + feed { item 'https://example.com/b', 'B', 'the body of B' } + })[0].items[0].description + + description.should include("ARTICLE 1\nTitle: \nURL: \n") + description.should include('the body of B') + end + end + + describe 'the joined item' do + let(:joined) { + join(config, AutomaticSpec.generate_pipeline { + feed { item 'https://example.com/a', 'A', 'the body of A' } + })[0].items[0] + } + + context 'with no title configured' do + let(:config) { {} } + + it 'takes the default title' do + joined.title.should == Automatic::Plugin::FilterJoin::DEFAULT_TITLE + end + + # There is no page this item points at, and the first article's URL would + # name a source for text that is not only from it. + it 'invents no permalink of its own' do + joined.link.should be_nil + end + end + + context 'with a title configured' do + let(:config) { { 'title' => 'Daily Digest' } } + + it 'takes the title the Recipe gives it' do + joined.title.should == 'Daily Digest' + end + end + end +end diff --git a/spec/plugins/filter/open_ai_spec.rb b/spec/plugins/filter/open_ai_spec.rb new file mode 100644 index 0000000..c4ac23b --- /dev/null +++ b/spec/plugins/filter/open_ai_spec.rb @@ -0,0 +1,278 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::OpenAI +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. + +require File.expand_path(File.dirname(__FILE__) + '../../../spec_helper') + +require 'filter/open_ai' +require 'json' +require 'net/http' + +# FilterOpenAI needs an OpenAI API token; doc/PLUGINS.md section 6.3 classifies +# it as Supported (external). Everything up to the request is verified here -- +# what is posted where, with which authentication, and what each answer means +# -- and no example reaches the service. Only OpenAI is exercised: the other AI +# filters have their own specs, and none of them stands in for another. +module OpenAISpec + module_function + + # A real response object, because the plugin decides what to do from the + # class Net::HTTP would have given it. + def response(klass, code, body) + response = klass.new('1.1', code, 'Status') + response.instance_variable_set(:@body, body) + response.instance_variable_set(:@read, true) + response + end + + def answered(text) + response(Net::HTTPOK, '200', JSON.generate( + 'output' => [ + { 'type' => 'reasoning', 'summary' => [] }, + { 'type' => 'message', 'content' => [{ 'type' => 'output_text', 'text' => text }] } + ] + )) + end + + def refused(klass, code, message) + response(klass, code, JSON.generate('error' => { 'message' => message })) + end +end + +describe Automatic::Plugin::FilterOpenAI do + let(:settings) { + { 'token' => 'test-token', 'model' => 'gpt-test', 'prompt' => 'Summarize this.', + 'retry' => 1, 'interval' => 0 } + } + + let(:posted) { [] } + let(:connections) { [] } + + def one_item + AutomaticSpec.generate_pipeline { + feed { item 'https://example.com/a', 'A', 'the body of A' } + } + end + + # Stands in for the network: records the connection and the request, answers + # with what the example queued, and opens nothing. The default suite reaches + # no network (doc/POLICY.md Invariant 6). + def serve(*responses) + requests = posted + opened = connections + Net::HTTP.stub(:start) { |*args, &block| + opened << args + http = double('http') + http.stub(:request) { |request| + requests << request + responses.shift + } + block.call(http) + } + end + + def run(config, pipeline) + Automatic::Plugin::FilterOpenAI.new(config, pipeline).run + end + + describe 'the request it builds' do + before { + serve(OpenAISpec.answered('a summary')) + run(settings, one_item) + } + + it 'posts to the Responses API over TLS, with the certificate verified' do + connections[0][0].should == 'api.openai.com' + connections[0][1].should == 443 + connections[0].last[:use_ssl].should == true + connections[0].last[:verify_mode].should == OpenSSL::SSL::VERIFY_PEER + posted[0].path.should == '/v1/responses' + end + + it 'sends the token as a bearer token' do + posted[0]['authorization'].should == 'Bearer test-token' + posted[0]['content-type'].should == 'application/json' + end + + it 'sends the model, the prompt as the instruction and the description as the input' do + JSON.parse(posted[0].body).should == { + 'model' => 'gpt-test', + 'instructions' => 'Summarize this.', + 'input' => 'the body of A' + } + end + end + + describe 'what it does to the pipeline' do + it 'replaces the description with the answer and leaves the rest alone' do + serve(OpenAISpec.answered('a summary')) + returned = run(settings, one_item) + + returned.should have(1).feed + returned[0].items.should have(1).item + returned[0].items[0].description.should == 'a summary' + returned[0].items[0].title.should == 'A' + returned[0].items[0].link.should == 'https://example.com/a' + end + + it 'asks once for each item' do + serve(OpenAISpec.answered('first summary'), OpenAISpec.answered('second summary')) + returned = run(settings, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', 'the body of A' + item 'https://example.com/b', 'B', 'the body of B' + } + }) + + posted.should have(2).requests + JSON.parse(posted[0].body)['input'].should == 'the body of A' + JSON.parse(posted[1].body)['input'].should == 'the body of B' + returned[0].items.map(&:description).should == ['first summary', 'second summary'] + end + + it 'ignores a feed that is nil' do + serve(OpenAISpec.answered('a summary')) + run(settings, [nil] + one_item).should have(2).feeds + posted.should have(1).request + end + + it 'sends nothing for an item with no description, and empties none' do + serve + returned = run(settings, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', '' + item 'https://example.com/b', 'B', ' ' + } + }) + + posted.should be_empty + returned[0].items.map(&:description).should == ['', ' '] + end + end + + describe 'when the request fails' do + it 'retries a server error and carries on' do + serve(OpenAISpec.refused(Net::HTTPInternalServerError, '500', 'server error'), + OpenAISpec.answered('a summary')) + returned = run(settings, one_item) + + posted.should have(2).requests + returned[0].items[0].description.should == 'a summary' + end + + it 'retries a rate limit' do + serve(OpenAISpec.refused(Net::HTTPTooManyRequests, '429', 'slow down'), + OpenAISpec.answered('a summary')) + run(settings, one_item)[0].items[0].description.should == 'a summary' + posted.should have(2).requests + end + + it 'gives up after the configured number of retries' do + serve(OpenAISpec.refused(Net::HTTPInternalServerError, '500', 'server error'), + OpenAISpec.refused(Net::HTTPInternalServerError, '500', 'server error')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterOpenAI::Error, /gave up/) + posted.should have(2).requests + end + + it 'does not retry a rejected request' do + serve(OpenAISpec.refused(Net::HTTPUnauthorized, '401', 'invalid api key')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterOpenAI::Error, /401/) + posted.should have(1).request + end + + it 'does not empty the description when the service fails' do + serve(OpenAISpec.refused(Net::HTTPUnauthorized, '401', 'invalid api key')) + pipeline = one_item + lambda { run(settings, pipeline) }.should raise_error(StandardError) + pipeline[0].items[0].description.should == 'the body of A' + end + end + + describe 'when the answer cannot be read' do + it 'raises on a body that is not JSON' do + serve(OpenAISpec.response(Net::HTTPOK, '200', 'not json at all')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterOpenAI::Error, /not JSON/) + posted.should have(1).request + end + + it 'raises on a body without the output it expects' do + serve(OpenAISpec.response(Net::HTTPOK, '200', JSON.generate('id' => 'resp_1'))) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterOpenAI::Error, /output array/) + end + + it 'raises rather than writing an empty description' do + serve(OpenAISpec.response(Net::HTTPOK, '200', JSON.generate( + 'output' => [{ 'type' => 'message', 'content' => [] }] + ))) + pipeline = one_item + lambda { run(settings, pipeline) }. + should raise_error(Automatic::Plugin::FilterOpenAI::Error, /no output text/) + pipeline[0].items[0].description.should == 'the body of A' + end + end + + describe 'the settings it requires' do + it 'refuses a Recipe with no token' do + lambda { run(settings.merge('token' => nil), one_item) }. + should raise_error(ArgumentError, /token/) + end + + it 'refuses a Recipe with no model' do + lambda { run(settings.merge('model' => ''), one_item) }. + should raise_error(ArgumentError, /model/) + end + + it 'refuses a Recipe with no prompt' do + lambda { run(settings.merge('prompt' => nil), one_item) }. + should raise_error(ArgumentError, /prompt/) + end + + it 'asks nothing before it has what it needs' do + serve + lambda { run(settings.merge('prompt' => nil), one_item) }.should raise_error(ArgumentError) + posted.should be_empty + end + end + + describe 'the credential' do + # doc/PLUGINS.md section 3.7: never logged, never in an exception message, + # never written into an item. + it 'reaches neither the log nor the error, on the path that fails' do + messages = [] + logger = double('logger') + %i[info warn error].each { |level| logger.stub(level) { |message| messages << message.to_s } } + original = Automatic::Log.logger + + serve(OpenAISpec.refused(Net::HTTPUnauthorized, '401', 'invalid api key')) + begin + Automatic::Log.logger = logger + Automatic::Log.level('info') + lambda { run(settings, one_item) }.should raise_error(StandardError) { |error| + error.message.should_not include('test-token') + } + ensure + Automatic::Log.level('none') + Automatic::Log.logger = original + end + + messages.should_not be_empty + messages.each { |message| message.should_not include('test-token') } + end + + it 'writes nothing of itself into the item' do + serve(OpenAISpec.answered('a summary')) + returned = run(settings, one_item) + returned[0].items[0].description.should_not include('test-token') + end + end +end diff --git a/spec/plugins/filter/sakura_ai_spec.rb b/spec/plugins/filter/sakura_ai_spec.rb new file mode 100644 index 0000000..373aa17 --- /dev/null +++ b/spec/plugins/filter/sakura_ai_spec.rb @@ -0,0 +1,283 @@ +# -*- coding: utf-8 -*- +# Name:: Automatic::Plugin::Filter::SakuraAI +# Author: id774 (More info: http://id774.net) +# Source Code:: https://github.com/id774/automaticruby +# License:: The GPL version 3, or LGPL version 3 (Dual License). +# Contact:: idnanashi@gmail.com +# Created:: Aug 17, 2026 +# Updated:: Aug 17, 2026 +# Copyright:: Copyright (c) 2012-2026 Automatic Ruby Developers. + +require File.expand_path(File.dirname(__FILE__) + '../../../spec_helper') + +require 'filter/sakura_ai' +require 'json' +require 'net/http' + +# FilterSakuraAI needs a Sakura AI Engine token; doc/PLUGINS.md section 6.3 +# classifies it as Supported (external). Everything up to the request is +# verified here -- what is posted where, with which authentication, and what +# each answer means -- and no example reaches the service. Only the Sakura AI +# Engine is exercised: the other AI filters have their own specs, and none of +# them stands in for another. +module SakuraAISpec + module_function + + # A real response object, because the plugin decides what to do from the + # class Net::HTTP would have given it. + def response(klass, code, body) + response = klass.new('1.1', code, 'Status') + response.instance_variable_set(:@body, body) + response.instance_variable_set(:@read, true) + response + end + + def answered(text) + response(Net::HTTPOK, '200', JSON.generate( + 'object' => 'chat.completion', + 'choices' => [ + { 'index' => 0, + 'message' => { 'role' => 'assistant', 'content' => text }, + 'finish_reason' => 'stop' } + ] + )) + end + + def refused(klass, code, message) + response(klass, code, JSON.generate('error' => { 'message' => message })) + end +end + +describe Automatic::Plugin::FilterSakuraAI do + let(:settings) { + { 'token' => 'test-token', 'model' => 'sakura-test', 'prompt' => 'Summarize this.', + 'retry' => 1, 'interval' => 0 } + } + + let(:posted) { [] } + let(:connections) { [] } + + def one_item + AutomaticSpec.generate_pipeline { + feed { item 'https://example.com/a', 'A', 'the body of A' } + } + end + + # Stands in for the network: records the connection and the request, answers + # with what the example queued, and opens nothing. The default suite reaches + # no network (doc/POLICY.md Invariant 6). + def serve(*responses) + requests = posted + opened = connections + Net::HTTP.stub(:start) { |*args, &block| + opened << args + http = double('http') + http.stub(:request) { |request| + requests << request + responses.shift + } + block.call(http) + } + end + + def run(config, pipeline) + Automatic::Plugin::FilterSakuraAI.new(config, pipeline).run + end + + describe 'the request it builds' do + before { + serve(SakuraAISpec.answered('a summary')) + run(settings, one_item) + } + + it "posts to the Sakura AI Engine's own endpoint, over verified TLS" do + connections[0][0].should == 'api.ai.sakura.ad.jp' + connections[0][1].should == 443 + connections[0].last[:use_ssl].should == true + connections[0].last[:verify_mode].should == OpenSSL::SSL::VERIFY_PEER + posted[0].path.should == '/v1/chat/completions' + end + + it 'sends the token as a bearer token' do + posted[0]['authorization'].should == 'Bearer test-token' + posted[0]['content-type'].should == 'application/json' + end + + it 'sends the model, the prompt as the system turn and the description as the user turn' do + JSON.parse(posted[0].body).should == { + 'model' => 'sakura-test', + 'messages' => [ + { 'role' => 'system', 'content' => 'Summarize this.' }, + { 'role' => 'user', 'content' => 'the body of A' } + ] + } + end + end + + describe 'what it does to the pipeline' do + it 'replaces the description with the answer and leaves the rest alone' do + serve(SakuraAISpec.answered('a summary')) + returned = run(settings, one_item) + + returned.should have(1).feed + returned[0].items.should have(1).item + returned[0].items[0].description.should == 'a summary' + returned[0].items[0].title.should == 'A' + returned[0].items[0].link.should == 'https://example.com/a' + end + + it 'asks once for each item' do + serve(SakuraAISpec.answered('first summary'), SakuraAISpec.answered('second summary')) + returned = run(settings, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', 'the body of A' + item 'https://example.com/b', 'B', 'the body of B' + } + }) + + posted.should have(2).requests + JSON.parse(posted[0].body)['messages'][1]['content'].should == 'the body of A' + JSON.parse(posted[1].body)['messages'][1]['content'].should == 'the body of B' + returned[0].items.map(&:description).should == ['first summary', 'second summary'] + end + + it 'ignores a feed that is nil' do + serve(SakuraAISpec.answered('a summary')) + run(settings, [nil] + one_item).should have(2).feeds + posted.should have(1).request + end + + it 'sends nothing for an item with no description, and empties none' do + serve + returned = run(settings, AutomaticSpec.generate_pipeline { + feed { + item 'https://example.com/a', 'A', '' + item 'https://example.com/b', 'B', ' ' + } + }) + + posted.should be_empty + returned[0].items.map(&:description).should == ['', ' '] + end + end + + describe 'when the request fails' do + it 'retries a server error and carries on' do + serve(SakuraAISpec.refused(Net::HTTPInternalServerError, '500', 'server error'), + SakuraAISpec.answered('a summary')) + returned = run(settings, one_item) + + posted.should have(2).requests + returned[0].items[0].description.should == 'a summary' + end + + it 'retries a rate limit' do + serve(SakuraAISpec.refused(Net::HTTPTooManyRequests, '429', 'too many requests'), + SakuraAISpec.answered('a summary')) + run(settings, one_item)[0].items[0].description.should == 'a summary' + posted.should have(2).requests + end + + it 'gives up after the configured number of retries' do + serve(SakuraAISpec.refused(Net::HTTPInternalServerError, '500', 'server error'), + SakuraAISpec.refused(Net::HTTPInternalServerError, '500', 'server error')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterSakuraAI::Error, /gave up/) + posted.should have(2).requests + end + + it 'does not retry a rejected request' do + serve(SakuraAISpec.refused(Net::HTTPUnauthorized, '401', 'invalid token')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterSakuraAI::Error, /401/) + posted.should have(1).request + end + + it 'does not empty the description when the service fails' do + serve(SakuraAISpec.refused(Net::HTTPUnauthorized, '401', 'invalid token')) + pipeline = one_item + lambda { run(settings, pipeline) }.should raise_error(StandardError) + pipeline[0].items[0].description.should == 'the body of A' + end + end + + describe 'when the answer cannot be read' do + it 'raises on a body that is not JSON' do + serve(SakuraAISpec.response(Net::HTTPOK, '200', 'not json at all')) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterSakuraAI::Error, /not JSON/) + posted.should have(1).request + end + + it 'raises on a body without the choice it expects' do + serve(SakuraAISpec.response(Net::HTTPOK, '200', JSON.generate('id' => 'cmpl_1'))) + lambda { run(settings, one_item) }. + should raise_error(Automatic::Plugin::FilterSakuraAI::Error, /without a choice/) + end + + it 'raises rather than writing an empty description' do + serve(SakuraAISpec.response(Net::HTTPOK, '200', JSON.generate( + 'choices' => [{ 'message' => { 'role' => 'assistant', 'content' => '' } }] + ))) + pipeline = one_item + lambda { run(settings, pipeline) }. + should raise_error(Automatic::Plugin::FilterSakuraAI::Error, /no content/) + pipeline[0].items[0].description.should == 'the body of A' + end + end + + describe 'the settings it requires' do + it 'refuses a Recipe with no token' do + lambda { run(settings.merge('token' => nil), one_item) }. + should raise_error(ArgumentError, /token/) + end + + it 'refuses a Recipe with no model' do + lambda { run(settings.merge('model' => ''), one_item) }. + should raise_error(ArgumentError, /model/) + end + + it 'refuses a Recipe with no prompt' do + lambda { run(settings.merge('prompt' => nil), one_item) }. + should raise_error(ArgumentError, /prompt/) + end + + it 'asks nothing before it has what it needs' do + serve + lambda { run(settings.merge('prompt' => nil), one_item) }.should raise_error(ArgumentError) + posted.should be_empty + end + end + + describe 'the credential' do + # doc/PLUGINS.md section 3.7: never logged, never in an exception message, + # never written into an item. + it 'reaches neither the log nor the error, on the path that fails' do + messages = [] + logger = double('logger') + %i[info warn error].each { |level| logger.stub(level) { |message| messages << message.to_s } } + original = Automatic::Log.logger + + serve(SakuraAISpec.refused(Net::HTTPUnauthorized, '401', 'invalid token')) + begin + Automatic::Log.logger = logger + Automatic::Log.level('info') + lambda { run(settings, one_item) }.should raise_error(StandardError) { |error| + error.message.should_not include('test-token') + } + ensure + Automatic::Log.level('none') + Automatic::Log.logger = original + end + + messages.should_not be_empty + messages.each { |message| message.should_not include('test-token') } + end + + it 'writes nothing of itself into the item' do + serve(SakuraAISpec.answered('a summary')) + returned = run(settings, one_item) + returned[0].items[0].description.should_not include('test-token') + end + end +end