move tokens to config

PATCH_README_EVENTSOURCE_SHOP_STATUS_FIX.md

@@ -0,0 +1,42 @@
+# RetrieX EventSource + Shop Status Fix
+
+Patch-only package.
+
+## Purpose
+
+This patch extends the previous stream robustness fix and addresses two issues:
+
+1. The chat now shows a visible reason when the Shopware shop/search API is unreachable or returns a system error.
+2. Reference-probe failures no longer block the actual primary Shopware search request.
+
+## Files
+
+- `public/assets/js/base.js`
+  - Uses native `EventSource` instead of a manually parsed `fetch().body.getReader()` stream.
+  - Starts a short-lived stream job with `POST /ask-jobs`, then opens `GET /ask-sse/{jobId}`.
+
+- `src/Controller/AskSseController.php`
+  - Adds `POST /ask-jobs`.
+  - Adds `GET /ask-sse/{jobId}` for native EventSource streaming.
+  - Keeps the old `POST /ask-sse` endpoint as backwards compatibility.
+  - Stores short-lived stream jobs under `var/stream_jobs`.
+
+- `src/Commerce/ShopSearchService.php`
+  - Reference-probe Store API failures are logged but no longer stop the primary search.
+  - The failure state is reset after a failed reference probe before the primary search starts.
+  - Retry without commerce history is skipped only after a real primary Store API system failure.
+
+- `src/Agent/AgentRunner.php`
+  - If the primary shop search has a Store API system failure, the chat displays:
+    `Shopsystem aktuell nicht erreichbar oder fehlerhaft. Grund: ...`
+  - Repair search is skipped after primary Store API system failures.
+
+## After installation
+
+Run:
+
+```bash
+bin/console cache:clear
+```
+
+If OPcache/PHP-FPM is active, reload PHP-FPM afterwards.

RETRIEX_AGENT_CONFIG_FIX_README.md

@@ -0,0 +1,32 @@
+# RetrieX Agent Config Centralization Fix
+
+This patch moves low-risk AgentRunner configuration values into `config/retriex/agent.yaml` while keeping the existing PHP fallback values in `AgentRunnerConfig`.
+
+## Scope
+
+Changed files:
+
+- `config/retriex/agent.yaml`
+- `src/Config/AgentRunnerConfig.php`
+- `RETRIEX_AGENT_CONFIG_FIX_README.md`
+
+## What was centralized
+
+- user-visible progress/status messages
+- error messages
+- source labels
+- small HTML templates used by the agent stream/output
+- Shopware query optimizer prompt text and rules
+
+## What was intentionally not changed
+
+- `PromptBuilderConfig` prompt wording/rules
+- retrieval scoring logic
+- shop matching logic
+- vocabulary/intent/search-repair logic from the previous stable centralization step
+
+## Safety notes
+
+The YAML values mirror the existing PHP fallback values. `AgentRunnerConfig` still contains the old defaults as fallbacks, so missing or invalid YAML values should not break runtime behavior.
+
+After applying the patch, clear the Symfony cache and re-run the known 1.4.2 regression prompts.

config/retriex/agent.yaml

@@ -1,8 +1,69 @@
-# Agent orchestration limits and user-visible source/progress labels.
+# Agent orchestration limits, user-visible status/source labels and Shopware query prompt wording.
-# Values mirror the current 1.4.2 defaults.
+# Values mirror the current stable defaults; PHP fallbacks remain in AgentRunnerConfig.
 parameters:
   retriex.agent.config:
     commerce_history_budget_chars: 1000
     product_search_knowledge_chunk_limit: 6
     advisory_product_search_knowledge_chunk_limit: 9
     optimized_shop_query_prefix_pattern: '/^(?:keywords?|suchquery|search\s*query|query)\s*:\s*/iu'
+
+    messages:
+      empty_prompt: '❌ Empty prompt.'
+      analyze_request: 'Ich analysiere deine Anfrage...'
+      check_internet_sources: 'Ich prüfe auf Internetquellen...'
+      retrieve_knowledge: 'Ich hole relevante Daten aus meinem RAG-Wissen...'
+      optimize_search: 'Ich optimiere die Recherche...'
+      fetch_search_data_template: 'Ich rufe Recherchedaten ab (type: %s)'
+      analyze_all_information: 'Ich analysiere alle Informationen...'
+      thinking_while_streaming: 'Denke nach...'
+      no_llm_data_received: '❌ Es wurden keine Daten vom LLM empfangen.'
+      generic_internal_error: '❌ Bei der Verarbeitung der Anfrage ist ein interner Fehler aufgetreten.'
+      debug_internal_error_prefix: '❌ Interner Fehler: '
+
+    source_labels:
+      external_url: 'Externe URL'
+      rag_knowledge: 'RAG Wissen'
+      conversation_history: 'Chatverlauf'
+      shop_system: 'Shopsystem'
+      extended_shop_search: 'Erweiterte Shopsuche'
+      used_sources_prefix: 'Genutzte Quellen: '
+      sources_prefix: 'Quellen: '
+
+    html:
+      source_badge_template: '<span class="badge bg-info text-black">%s</span>'
+      error_template: |
+        <div class="retriex-alert retriex-alert--error"><div class="retriex-alert__icon">❌</div><div class="retriex-alert__content"><div class="retriex-alert__title">Hinweis</div><div class="retriex-alert__text">%s</div></div></div>
+      think_template: |
+        <span class="text-info think">%s</span>
+      info_template: "\n\n<span class=\"text-info fw-bolder\">%s</span>\n"
+      debug_template: "\n\nDEBUG: <code>%s</code>\n"
+
+    shop_prompt:
+      intro: 'Generate a short search query for Shopware 6 from the following user input text.'
+      output_format_block: |-
+        Output format:
+        Keyword1 Keyword2 Keyword3
+      recent_conversation_context_label: 'RECENT CONVERSATION CONTEXT'
+      current_user_input_label: 'CURRENT USER INPUT'
+      rules:
+        - '- Output only the final search query.'
+        - '- Always convert relevant search terms to their singular form.'
+        - '- No introduction, no explanation, no quotation marks.'
+        - '- Use only shop-relevant search terms from the user input for a shop search.'
+        - '- Maximum 6 search terms, preferably fewer.'
+        - '- Remove filler words, polite phrases, and irrelevant words.'
+        - '- Preserve product names, brands, model numbers, and compound terms exactly if they are relevant.'
+        - '- Numbers that belong to a product name or model must be preserved (e.g. Indikator 300, Testomat 808, Testomat 2000).'
+        - '- Separate terms using spaces only.'
+        - '- If a relevant product name is present, it must be placed at the beginning of the final search query.'
+        - '- Try to always identify all products mentioned in the user input text, even in long prompts.'
+        - '- Look for terms such as Testomat, Horiba, Tritromat, or words like indicator.'
+        - '- If the current user input is vague or referential, use the recent conversation context only as support.'
+        - '- Do not output words that only describe conversation flow, such as "same", "again", "also", or "like above".'
+      conversation_context_rules:
+        - '- The current user input has highest priority.'
+        - '- Use the recent conversation context only to resolve omitted references.'
+        - '- Use it only for product carry-over, brand carry-over, model carry-over, or variant follow-ups.'
+        - '- Do not revive older products unless the current user input clearly refers to them.'
+        - '- If the current input starts a new topic, ignore older product context.'
+        - '- Prefer the most recent product reference over older ones.'

public/assets/styles/base.css

@@ -378,7 +378,7 @@ span.think {
 }
 
 .think {
-    display: inline-block;
+    display: block;
     color: rgba(255, 255, 255, 0.72);
     background-image: linear-gradient(
             100deg,

src/Config/AgentRunnerConfig.php

@@ -36,136 +36,180 @@ final class AgentRunnerConfig
 
     public function getOptimizedShopQueryTrimCharacters(): string
     {
-        return " \t\n\r\0\x0B\"'`";
+        return $this->getString('optimized_shop_query_trim_characters', " \t\n\r\0\x0B\"'`");
     }
 
     private function getInt(string $key, int $default): int
     {
-        $value = $this->config[$key] ?? $default;
+        $value = $this->value($key, $default);
 
         return is_numeric($value) ? (int) $value : $default;
     }
 
     private function getString(string $key, string $default): string
     {
-        $value = $this->config[$key] ?? $default;
+        $value = $this->value($key, $default);
 
         return is_string($value) && $value !== '' ? $value : $default;
     }
 
+    /**
+     * @param string[] $default
+     * @return string[]
+     */
+    private function getStringList(string $key, array $default): array
+    {
+        $value = $this->value($key, $default);
+
+        if (!is_array($value)) {
+            return $default;
+        }
+
+        $out = [];
+
+        foreach ($value as $item) {
+            if (!is_scalar($item)) {
+                continue;
+            }
+
+            $item = trim((string) $item);
+
+            if ($item !== '') {
+                $out[] = $item;
+            }
+        }
+
+        return $out !== [] ? $out : $default;
+    }
+
+    private function value(string $key, mixed $default): mixed
+    {
+        $current = $this->config;
+
+        foreach (explode('.', $key) as $segment) {
+            if (!is_array($current) || !array_key_exists($segment, $current)) {
+                return $default;
+            }
+
+            $current = $current[$segment];
+        }
+
+        return $current;
+    }
+
     public function getEmptyPromptMessage(): string
     {
-        return '❌ Empty prompt.';
+        return $this->getString('messages.empty_prompt', '❌ Empty prompt.');
     }
 
     public function getAnalyzeRequestMessage(): string
     {
-        return 'Ich analysiere deine Anfrage...';
+        return $this->getString('messages.analyze_request', 'Ich analysiere deine Anfrage...');
     }
 
     public function getCheckInternetSourcesMessage(): string
     {
-        return 'Ich prüfe auf Internetquellen...';
+        return $this->getString('messages.check_internet_sources', 'Ich prüfe auf Internetquellen...');
     }
 
     public function getRetrieveKnowledgeMessage(): string
     {
-        return 'Ich hole relevante Daten aus meinem RAG-Wissen...';
+        return $this->getString('messages.retrieve_knowledge', 'Ich hole relevante Daten aus meinem RAG-Wissen...');
     }
 
     public function getOptimizeSearchMessage(): string
     {
-        return 'Ich optimiere die Recherche...';
+        return $this->getString('messages.optimize_search', 'Ich optimiere die Recherche...');
     }
 
     public function getFetchSearchDataMessageTemplate(): string
     {
-        return 'Ich rufe Recherchedaten ab (type: %s)';
+        return $this->getString('messages.fetch_search_data_template', 'Ich rufe Recherchedaten ab (type: %s)');
     }
 
     public function getAnalyzeAllInformationMessage(): string
     {
-        return 'Ich analysiere alle Informationen...';
+        return $this->getString('messages.analyze_all_information', 'Ich analysiere alle Informationen...');
     }
 
     public function getThinkingWhileStreamingMessage(): string
     {
-        return 'Denke nach...';
+        return $this->getString('messages.thinking_while_streaming', 'Denke nach...');
     }
 
     public function getNoLlmDataReceivedMessage(): string
     {
-        return '❌ Es wurden keine Daten vom LLM empfangen.';
+        return $this->getString('messages.no_llm_data_received', '❌ Es wurden keine Daten vom LLM empfangen.');
     }
 
     public function getGenericInternalErrorMessage(): string
     {
-        return '❌ Bei der Verarbeitung der Anfrage ist ein interner Fehler aufgetreten.';
+        return $this->getString('messages.generic_internal_error', '❌ Bei der Verarbeitung der Anfrage ist ein interner Fehler aufgetreten.');
     }
 
     public function getDebugInternalErrorPrefix(): string
     {
-        return '❌ Interner Fehler: ';
+        return $this->getString('messages.debug_internal_error_prefix', '❌ Interner Fehler: ');
     }
 
     public function getExternalUrlSourceLabel(): string
     {
-        return 'Externe URL';
+        return $this->getString('source_labels.external_url', 'Externe URL');
     }
 
     public function getRagKnowledgeSourceLabel(): string
     {
-        return 'RAG Wissen';
+        return $this->getString('source_labels.rag_knowledge', 'RAG Wissen');
     }
 
     public function getConversationHistorySourceLabel(): string
     {
-        return 'Chatverlauf';
+        return $this->getString('source_labels.conversation_history', 'Chatverlauf');
     }
 
     public function getShopSystemSourceLabel(): string
     {
-        return 'Shopsystem';
+        return $this->getString('source_labels.shop_system', 'Shopsystem');
     }
 
     public function getExtendedShopSearchSourceLabel(): string
     {
-        return 'Erweiterte Shopsuche';
+        return $this->getString('source_labels.extended_shop_search', 'Erweiterte Shopsuche');
     }
 
     public function getUsedSourcesPrefix(): string
     {
-        return 'Genutzte Quellen: ';
+        return $this->getString('source_labels.used_sources_prefix', 'Genutzte Quellen: ');
     }
 
     public function getSourcesPrefix(): string
     {
-        return 'Quellen: ';
+        return $this->getString('source_labels.sources_prefix', 'Quellen: ');
     }
 
     public function getSourceBadgeHtmlTemplate(): string
     {
-        return '<span class="badge bg-info text-black">%s</span>';
+        return $this->getString('html.source_badge_template', '<span class="badge bg-info text-black">%s</span>');
     }
 
     public function getErrorHtmlTemplate(): string
     {
-        return '<div class="retriex-alert retriex-alert--error"><div class="retriex-alert__icon">❌</div><div class="retriex-alert__content"><div class="retriex-alert__title">Hinweis</div><div class="retriex-alert__text">%s</div></div></div>' . "\n";
+        return $this->getString('html.error_template', '<div class="retriex-alert retriex-alert--error"><div class="retriex-alert__icon">❌</div><div class="retriex-alert__content"><div class="retriex-alert__title">Hinweis</div><div class="retriex-alert__text">%s</div></div></div>' . "\n");
     }
 
     public function getThinkHtmlTemplate(): string
     {
-        return '<span class="text-info think">%s</span>' . "\n";
+        return $this->getString('html.think_template', '<span class="text-info think">%s</span>' . "\n");
     }
 
     public function getInfoHtmlTemplate(): string
     {
-        return "\n\n" . '<span class="text-info fw-bolder">%s</span>' . "\n";
+        return $this->getString('html.info_template', "\n\n" . '<span class="text-info fw-bolder">%s</span>' . "\n");
     }
 
     public function getDebugHtmlTemplate(): string
     {
-        return "\n\nDEBUG: <code>%s</code>\n";
+        return $this->getString('html.debug_template', "\n\nDEBUG: <code>%s</code>\n");
     }
 
     public function getShopPrompt(string $prompt, string $commerceHistoryContext = ''): string
@@ -200,7 +244,7 @@ final class AgentRunnerConfig
      */
     public function getShopPromptRules(): array
     {
-        return [
+        return $this->getStringList('shop_prompt.rules', [
             '- Output only the final search query.',
             '- Always convert relevant search terms to their singular form.',
             '- No introduction, no explanation, no quotation marks.',
@@ -215,7 +259,7 @@ final class AgentRunnerConfig
             '- Look for terms such as Testomat, Horiba, Tritromat, or words like indicator.',
             '- If the current user input is vague or referential, use the recent conversation context only as support.',
             '- Do not output words that only describe conversation flow, such as "same", "again", "also", or "like above".',
-        ];
+        ]);
     }
 
     /**
@@ -223,34 +267,34 @@ final class AgentRunnerConfig
      */
     public function getConversationContextRules(): array
     {
-        return [
+        return $this->getStringList('shop_prompt.conversation_context_rules', [
             '- The current user input has highest priority.',
             '- Use the recent conversation context only to resolve omitted references.',
             '- Use it only for product carry-over, brand carry-over, model carry-over, or variant follow-ups.',
             '- Do not revive older products unless the current user input clearly refers to them.',
             '- If the current input starts a new topic, ignore older product context.',
             '- Prefer the most recent product reference over older ones.',
-        ];
+        ]);
     }
 
     public function getShopPromptIntro(): string
     {
-        return 'Generate a short search query for Shopware 6 from the following user input text.';
+        return $this->getString('shop_prompt.intro', 'Generate a short search query for Shopware 6 from the following user input text.');
     }
 
     public function getShopPromptOutputFormatBlock(): string
     {
-        return "Output format:\nKeyword1 Keyword2 Keyword3";
+        return $this->getString('shop_prompt.output_format_block', "Output format:\nKeyword1 Keyword2 Keyword3");
     }
 
     public function getRecentConversationContextLabel(): string
     {
-        return 'RECENT CONVERSATION CONTEXT';
+        return $this->getString('shop_prompt.recent_conversation_context_label', 'RECENT CONVERSATION CONTEXT');
     }
 
     public function getCurrentUserInputLabel(): string
     {
-        return 'CURRENT USER INPUT';
+        return $this->getString('shop_prompt.current_user_input_label', 'CURRENT USER INPUT');
     }
 
     private function buildRulesBlock(array $rules, string $headline = 'Rules:'): string