harden information getter services and optimize user msg
This commit is contained in:
35
RETRIEX_NO_LLM_FALLBACK_ESCALATION_FIX_README.md
Normal file
35
RETRIEX_NO_LLM_FALLBACK_ESCALATION_FIX_README.md
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
# RetrieX No-LLM Fallback & Eskalationslogik Fix
|
||||||
|
|
||||||
|
Patch-only ZIP auf Basis der aktuell hochgeladenen `rag-inprogress.zip`.
|
||||||
|
|
||||||
|
## Ziel
|
||||||
|
|
||||||
|
RetrieX soll auch ohne LLM bzw. bei fehlender LLM-Antwort keine harte oder falsche Negativaussage ausgeben, sondern den Datenzustand transparent und deterministisch darstellen.
|
||||||
|
|
||||||
|
## Enthaltene Änderungen
|
||||||
|
|
||||||
|
- deterministische No-LLM-Fallbackantworten in `AgentRunner`
|
||||||
|
- Fallback greift nur, wenn das LLM keine Antworttokens liefert oder vor dem ersten Antworttoken ausfällt
|
||||||
|
- keine Interpretation technischer Eignung aus Shopdaten im No-LLM-Modus
|
||||||
|
- Shop-only Treffer werden als Shopdaten gekennzeichnet: technische Eignung bitte prüfen
|
||||||
|
- keine harte Negativaussage bei leeren Shop-/RAG-Treffern
|
||||||
|
- getrennte Meldungen für:
|
||||||
|
- Shop-Treffer ohne RAG-Fachwissen
|
||||||
|
- Shop-Treffer mit RAG-/Kontexttreffern
|
||||||
|
- keine Shop-Treffer
|
||||||
|
- Shop nicht erreichbar
|
||||||
|
- RAG-Treffer vorhanden, aber keine No-LLM-Synthese
|
||||||
|
- keine belastbaren Daten
|
||||||
|
- No-concrete-Shop-Query-Meldung entschärft und in die History geschrieben
|
||||||
|
- Prompt-Regeln für Unsicherheits-/Fallbackstufen bleiben zusätzlich für den LLM-Modus enthalten
|
||||||
|
|
||||||
|
## Sicherheitsentscheidung
|
||||||
|
|
||||||
|
Die No-LLM-Schicht synthetisiert bewusst keine fachlichen Aussagen aus RAG-Chunks und leitet keine technische Eignung aus Shopdaten ab. Sie listet nur belegte Shop-Metadaten und den sicheren nächsten Schritt.
|
||||||
|
|
||||||
|
## Validierung
|
||||||
|
|
||||||
|
- `php -l` für alle geänderten PHP-Dateien: OK
|
||||||
|
- YAML-Parsing für `config/retriex/prompt.yaml` und `config/retriex/agent.yaml`: OK
|
||||||
|
|
||||||
|
Die Symfony-Regression kann in diesem Archiv nicht ausgeführt werden, weil `vendor/autoload.php` nicht im ZIP enthalten ist.
|
||||||
@@ -13,7 +13,7 @@ parameters:
|
|||||||
check_internet_sources: 'Ich prüfe auf Internetquellen...'
|
check_internet_sources: 'Ich prüfe auf Internetquellen...'
|
||||||
retrieve_knowledge: 'Ich hole relevante Daten aus meinem RAG-Wissen...'
|
retrieve_knowledge: 'Ich hole relevante Daten aus meinem RAG-Wissen...'
|
||||||
optimize_search: 'Ich optimiere die Recherche...'
|
optimize_search: 'Ich optimiere die Recherche...'
|
||||||
no_concrete_shop_query: 'Ich habe keine konkrete Shop-Suchanfrage erkannt. Bitte nenne das Produkt, Zubehör oder die Artikelnummer.'
|
no_concrete_shop_query: 'Ich kann die Shop-Suche noch nicht belastbar auflösen. Bitte nenne das Produkt, den Messparameter oder das Zubehör etwas konkreter.'
|
||||||
fetch_search_data_template: 'Ich rufe Recherchedaten ab (type: %s)'
|
fetch_search_data_template: 'Ich rufe Recherchedaten ab (type: %s)'
|
||||||
analyze_all_information: 'Ich analysiere alle Informationen...'
|
analyze_all_information: 'Ich analysiere alle Informationen...'
|
||||||
thinking_while_streaming: 'Denke nach...'
|
thinking_while_streaming: 'Denke nach...'
|
||||||
@@ -21,6 +21,19 @@ parameters:
|
|||||||
generic_internal_error: '❌ Bei der Verarbeitung der Anfrage ist ein interner Fehler aufgetreten.'
|
generic_internal_error: '❌ Bei der Verarbeitung der Anfrage ist ein interner Fehler aufgetreten.'
|
||||||
debug_internal_error_prefix: '❌ Interner Fehler: '
|
debug_internal_error_prefix: '❌ Interner Fehler: '
|
||||||
|
|
||||||
|
no_llm_fallback:
|
||||||
|
max_shop_results: 5
|
||||||
|
messages:
|
||||||
|
shop_only: 'Ich finde dazu im RAG-Wissen keine belastbare Fachinformation. Aus den Shopdaten ergeben sich folgende Treffer; technische Eignung bitte prüfen:'
|
||||||
|
shop_with_knowledge: 'Es liegen RAG-/Kontexttreffer und Shopdaten vor. Ohne LLM leite ich daraus keine technische Eignung ab. Die Shopdaten zeigen folgende Treffer; technische Eignung bitte prüfen:'
|
||||||
|
escalation: 'Für eine verbindliche Produktauswahl sollte der konkrete Anwendungsfall durch Vertrieb oder Support geprüft werden.'
|
||||||
|
knowledge_only: 'Ich habe Treffer im RAG-Wissen gefunden, aber ohne LLM kann ich daraus keine belastbare fachliche Antwort synthetisieren. Ich gebe deshalb keine sichere Produktaussage aus. Bitte aktiviere das LLM oder konkretisiere die Frage für eine gezielte Prüfung.'
|
||||||
|
no_data: 'Ich finde dazu keine belastbaren Daten in den vorliegenden Quellen. Bitte nenne Produkt, Messparameter, Zubehör oder Anwendungsfall genauer.'
|
||||||
|
no_shop_results_with_knowledge: 'Ich finde RAG-/Kontexttreffer, aber keine passenden Shop-Treffer zur aktuellen Suchanfrage. Das ist keine Aussage, dass es das Produkt nicht gibt. Ohne LLM gebe ich keine technische Negativaussage aus; bitte prüfe den Suchbegriff oder den Anwendungsfall gezielter.'
|
||||||
|
no_shop_results_no_knowledge: 'Ich finde weder belastbares RAG-Wissen noch passende Shop-Treffer zur aktuellen Suchanfrage. Das ist keine sichere Negativaussage. Bitte nenne Produkt, Messparameter oder Zubehör konkreter.'
|
||||||
|
shop_unavailable_with_knowledge: 'Live-Shopdaten konnten nicht geladen werden. Ohne Shop-Check treffe ich keine Aussage zu aktueller Verfügbarkeit, Preis oder Shop-Portfolio. Vorhandenes RAG-Wissen darf nur als fachlicher Kontext verstanden werden.'
|
||||||
|
shop_unavailable_no_knowledge: 'Live-Shopdaten konnten nicht geladen werden und ich finde kein belastbares RAG-Wissen. Ich kann daraus keine verlässliche Produkt- oder Verfügbarkeitsaussage ableiten.'
|
||||||
|
|
||||||
source_labels:
|
source_labels:
|
||||||
external_url: 'Externe URL'
|
external_url: 'Externe URL'
|
||||||
rag_knowledge: 'RAG Wissen'
|
rag_knowledge: 'RAG Wissen'
|
||||||
|
|||||||
@@ -47,6 +47,7 @@ parameters:
|
|||||||
conversation_context_label: CONVERSATION CONTEXT (contextual only)
|
conversation_context_label: CONVERSATION CONTEXT (contextual only)
|
||||||
shop_search_query_label: SHOP SEARCH QUERY
|
shop_search_query_label: SHOP SEARCH QUERY
|
||||||
output_priority_label: OUTPUT PRIORITY
|
output_priority_label: OUTPUT PRIORITY
|
||||||
|
fallback_escalation_label: FALLBACK AND ESCALATION RULES
|
||||||
response_format_label: RESPONSE FORMAT RULES
|
response_format_label: RESPONSE FORMAT RULES
|
||||||
language_rules_label: LANGUAGE RULES
|
language_rules_label: LANGUAGE RULES
|
||||||
fact_grounding_rules_label: FACT GROUNDING RULES
|
fact_grounding_rules_label: FACT GROUNDING RULES
|
||||||
@@ -72,6 +73,36 @@ parameters:
|
|||||||
- '- If one source chunk contains both the best matching value and nearby comparison values, use the nearby values only as context and do not include them unless the user asks for comparison or alternatives.'
|
- '- If one source chunk contains both the best matching value and nearby comparison values, use the nearby values only as context and do not include them unless the user asks for comparison or alternatives.'
|
||||||
- '- For lowest/highest/minimum/maximum questions, answer only the requested extreme value and the product/device explicitly connected to it.'
|
- '- For lowest/highest/minimum/maximum questions, answer only the requested extreme value and the product/device explicitly connected to it.'
|
||||||
- '- Do not add runner-up products, second-lowest values, adjacent ranges, broader tables, or explanatory comparisons unless explicitly requested.'
|
- '- Do not add runner-up products, second-lowest values, adjacent ranges, broader tables, or explanatory comparisons unless explicitly requested.'
|
||||||
|
fallback_escalation:
|
||||||
|
state_line_template: '- Internal confidence state: {state}.'
|
||||||
|
base_rules:
|
||||||
|
- '- Prefer transparent uncertainty over a confident but unsupported answer.'
|
||||||
|
- '- Never present missing or weak evidence as proof that a product, value, accessory, or suitability does not exist.'
|
||||||
|
- '- A negative answer is allowed only when the provided sources explicitly support that negative finding for the asked scope.'
|
||||||
|
- '- If several products, parameters, or accessories could match, ask one focused clarification question instead of guessing.'
|
||||||
|
- '- For risky or binding product selection, state that sales or support should verify the application before a final selection.'
|
||||||
|
without_shop_check_rules:
|
||||||
|
- '- If the question is product-related and no live shop check was performed in this run, do not make a portfolio-wide negative statement such as "there is no product".'
|
||||||
|
- '- Phrase missing evidence narrowly, for example: "Im RAG-Wissen finde ich dazu keine belastbare Information."'
|
||||||
|
- '- If useful, say that a shop search can be used to look for matching products, but do not claim shop results were checked unless they are present in the prompt.'
|
||||||
|
states:
|
||||||
|
sicher_beantwortbar:
|
||||||
|
- '- The retrieved factual knowledge or user-provided URL content is sufficient for the core answer. Answer directly, but do not exceed the provided facts.'
|
||||||
|
wahrscheinlich_beantwortbar:
|
||||||
|
- '- Retrieved knowledge and shop data are both available. Use retrieved knowledge for technical suitability and shop data for current commercial details.'
|
||||||
|
- '- If the two source types do not clearly refer to the same product identity, separate the technical answer from commercial shop hits.'
|
||||||
|
nur_shop_treffer_kein_belastbares_fachwissen:
|
||||||
|
- '- Start the answer by making the fallback clear: "Aus den Shopdaten ergeben sich folgende Treffer; technische Eignung bitte prüfen."'
|
||||||
|
- '- Do not present shop-only matches as verified technical suitability unless the shop text explicitly states that suitability.'
|
||||||
|
- '- Do not say that RAG knowledge confirms the result. Say that no belastbares RAG-Fachwissen was available for this selection.'
|
||||||
|
keine_belastbaren_daten:
|
||||||
|
- '- State that no reliable information was found in the provided RAG knowledge, URL content, or shop results.'
|
||||||
|
- '- Do not answer with "gibt es nicht". Use narrow wording such as "Ich finde dazu keine belastbaren Daten in den vorliegenden Quellen."'
|
||||||
|
- '- Ask one focused clarification question if a parameter, product family, accessory type, or application context would make the search answerable.'
|
||||||
|
shopdaten_nicht_verfuegbar:
|
||||||
|
- '- State that live shop data could not be loaded and answer only from retrieved knowledge or URL content if available.'
|
||||||
|
- '- Do not draw negative conclusions about current product availability, price, or shop portfolio while the shop is unavailable.'
|
||||||
|
|
||||||
response_format:
|
response_format:
|
||||||
base_rules:
|
base_rules:
|
||||||
- '- Keep normal spacing between all words. Never fuse words together.'
|
- '- Keep normal spacing between all words. Never fuse words together.'
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ declare(strict_types=1);
|
|||||||
|
|
||||||
namespace App\Agent;
|
namespace App\Agent;
|
||||||
|
|
||||||
|
use App\Commerce\Dto\ShopProductResult;
|
||||||
use App\Commerce\SearchRepairService;
|
use App\Commerce\SearchRepairService;
|
||||||
use App\Commerce\ShopSearchService;
|
use App\Commerce\ShopSearchService;
|
||||||
use App\Config\AgentRunnerConfig;
|
use App\Config\AgentRunnerConfig;
|
||||||
@@ -62,6 +63,7 @@ final readonly class AgentRunner
|
|||||||
$attemptedShopRepair = false;
|
$attemptedShopRepair = false;
|
||||||
$usedShopRepair = false;
|
$usedShopRepair = false;
|
||||||
$shopRepairQueries = [];
|
$shopRepairQueries = [];
|
||||||
|
$shopSearchAttempted = false;
|
||||||
$primaryShopSearchHadSystemFailure = false;
|
$primaryShopSearchHadSystemFailure = false;
|
||||||
$historyNotices = [];
|
$historyNotices = [];
|
||||||
|
|
||||||
@@ -141,11 +143,18 @@ final readonly class AgentRunner
|
|||||||
'hasRequestContextHint' => trim($requestContextHint) !== '',
|
'hasRequestContextHint' => trim($requestContextHint) !== '',
|
||||||
]);
|
]);
|
||||||
|
|
||||||
|
$noConcreteShopQueryMessage = $this->agentRunnerConfig->getNoConcreteShopQueryMessage();
|
||||||
|
|
||||||
yield $this->systemMsg(
|
yield $this->systemMsg(
|
||||||
$this->agentRunnerConfig->getNoConcreteShopQueryMessage(),
|
$noConcreteShopQueryMessage,
|
||||||
'info'
|
'info'
|
||||||
);
|
);
|
||||||
|
|
||||||
|
$this->contextService->appendHistory(
|
||||||
|
$userId,
|
||||||
|
$prompt,
|
||||||
|
$this->plainTextFromHtml($noConcreteShopQueryMessage)
|
||||||
|
);
|
||||||
|
|
||||||
return;
|
return;
|
||||||
} else {
|
} else {
|
||||||
@@ -185,6 +194,7 @@ final readonly class AgentRunner
|
|||||||
'think'
|
'think'
|
||||||
);
|
);
|
||||||
|
|
||||||
|
$shopSearchAttempted = true;
|
||||||
$primaryShopResults = $this->searchShop(
|
$primaryShopResults = $this->searchShop(
|
||||||
$shopSearchQuery,
|
$shopSearchQuery,
|
||||||
$commerceIntent,
|
$commerceIntent,
|
||||||
@@ -276,7 +286,9 @@ final readonly class AgentRunner
|
|||||||
knowledgeChunks: $knowledgeChunks,
|
knowledgeChunks: $knowledgeChunks,
|
||||||
shopResults: $shopResults,
|
shopResults: $shopResults,
|
||||||
fullContext: $forceFullContext,
|
fullContext: $forceFullContext,
|
||||||
swagFullOutPut: $optimizedShopQuery
|
swagFullOutPut: $optimizedShopQuery,
|
||||||
|
commerceSearchAttempted: $shopSearchAttempted,
|
||||||
|
shopSearchHadSystemFailure: $primaryShopSearchHadSystemFailure
|
||||||
);
|
);
|
||||||
|
|
||||||
if ($this->debug && $this->logPrompt) {
|
if ($this->debug && $this->logPrompt) {
|
||||||
@@ -292,6 +304,7 @@ final readonly class AgentRunner
|
|||||||
'attemptedShopRepair' => $attemptedShopRepair,
|
'attemptedShopRepair' => $attemptedShopRepair,
|
||||||
'usedShopRepair' => $usedShopRepair,
|
'usedShopRepair' => $usedShopRepair,
|
||||||
'shopRepairQueries' => $shopRepairQueries,
|
'shopRepairQueries' => $shopRepairQueries,
|
||||||
|
'shopSearchAttempted' => $shopSearchAttempted,
|
||||||
]);
|
]);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -312,7 +325,18 @@ final readonly class AgentRunner
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
$fullOutput = yield from $this->streamFinalAnswer($finalPrompt);
|
$noLlmFallbackAnswer = $this->buildNoLlmFallbackAnswer(
|
||||||
|
prompt: $prompt,
|
||||||
|
urlContent: $urlContent,
|
||||||
|
knowledgeChunks: $knowledgeChunks,
|
||||||
|
shopResults: $shopResults,
|
||||||
|
commerceIntent: $commerceIntent,
|
||||||
|
shopSearchAttempted: $shopSearchAttempted,
|
||||||
|
shopSearchHadSystemFailure: $primaryShopSearchHadSystemFailure,
|
||||||
|
shopSearchFailureReason: $primaryShopSearchFailureReason ?? null
|
||||||
|
);
|
||||||
|
|
||||||
|
$fullOutput = yield from $this->streamFinalAnswer($finalPrompt, $noLlmFallbackAnswer);
|
||||||
|
|
||||||
if ($sources !== []) {
|
if ($sources !== []) {
|
||||||
yield $this->emitSources(
|
yield $this->emitSources(
|
||||||
@@ -345,6 +369,7 @@ final readonly class AgentRunner
|
|||||||
'attemptedShopRepair' => $attemptedShopRepair,
|
'attemptedShopRepair' => $attemptedShopRepair,
|
||||||
'usedShopRepair' => $usedShopRepair,
|
'usedShopRepair' => $usedShopRepair,
|
||||||
'shopRepairQueries' => $shopRepairQueries,
|
'shopRepairQueries' => $shopRepairQueries,
|
||||||
|
'shopSearchAttempted' => $shopSearchAttempted,
|
||||||
'primaryShopSearchHadSystemFailure' => $primaryShopSearchHadSystemFailure,
|
'primaryShopSearchHadSystemFailure' => $primaryShopSearchHadSystemFailure,
|
||||||
'primaryShopSearchFailureReason' => $primaryShopSearchFailureReason ?? null,
|
'primaryShopSearchFailureReason' => $primaryShopSearchFailureReason ?? null,
|
||||||
'knowledgeChunkCount' => count($knowledgeChunks),
|
'knowledgeChunkCount' => count($knowledgeChunks),
|
||||||
@@ -1178,7 +1203,7 @@ final readonly class AgentRunner
|
|||||||
/**
|
/**
|
||||||
* @return Generator<int, string, mixed, string>
|
* @return Generator<int, string, mixed, string>
|
||||||
*/
|
*/
|
||||||
private function streamFinalAnswer(string $finalPrompt): Generator
|
private function streamFinalAnswer(string $finalPrompt, string $noLlmFallbackAnswer = ''): Generator
|
||||||
{
|
{
|
||||||
$fullOutput = '';
|
$fullOutput = '';
|
||||||
$thinkingNoticeShown = false;
|
$thinkingNoticeShown = false;
|
||||||
@@ -1189,6 +1214,7 @@ final readonly class AgentRunner
|
|||||||
yield $this->systemMsg($this->agentRunnerConfig->getThinkingWhileStreamingMessage(), 'think');
|
yield $this->systemMsg($this->agentRunnerConfig->getThinkingWhileStreamingMessage(), 'think');
|
||||||
$thinkingNoticeShown = true;
|
$thinkingNoticeShown = true;
|
||||||
|
|
||||||
|
try {
|
||||||
foreach ($this->ollamaClient->stream($finalPrompt) as $token) {
|
foreach ($this->ollamaClient->stream($finalPrompt) as $token) {
|
||||||
if (!is_string($token)) {
|
if (!is_string($token)) {
|
||||||
continue;
|
continue;
|
||||||
@@ -1212,17 +1238,194 @@ final readonly class AgentRunner
|
|||||||
yield $this->systemMsg($chunk, 'answer');
|
yield $this->systemMsg($chunk, 'answer');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
} catch (Throwable $e) {
|
||||||
|
$noLlmFallbackAnswer = trim($noLlmFallbackAnswer);
|
||||||
|
|
||||||
|
if ($noLlmFallbackAnswer === '' || $fullOutput !== '') {
|
||||||
|
throw $e;
|
||||||
|
}
|
||||||
|
|
||||||
|
$this->agentLogger->warning('LLM stream failed before answer tokens, using deterministic no-LLM fallback answer', [
|
||||||
|
'exception' => $e,
|
||||||
|
]);
|
||||||
|
|
||||||
|
$fullOutput = $noLlmFallbackAnswer;
|
||||||
|
yield $this->systemMsg($noLlmFallbackAnswer, 'answer');
|
||||||
|
|
||||||
|
return $fullOutput;
|
||||||
|
}
|
||||||
|
|
||||||
$finalChunk = $chunker->flush();
|
$finalChunk = $chunker->flush();
|
||||||
if ($finalChunk !== null) {
|
if ($finalChunk !== null) {
|
||||||
yield $this->systemMsg($finalChunk, 'answer');
|
yield $this->systemMsg($finalChunk, 'answer');
|
||||||
} elseif ($fullOutput === '') {
|
} elseif ($fullOutput === '') {
|
||||||
|
$noLlmFallbackAnswer = trim($noLlmFallbackAnswer);
|
||||||
|
|
||||||
|
if ($noLlmFallbackAnswer !== '') {
|
||||||
|
$fullOutput = $noLlmFallbackAnswer;
|
||||||
|
yield $this->systemMsg($noLlmFallbackAnswer, 'answer');
|
||||||
|
} else {
|
||||||
yield $this->systemMsg($this->agentRunnerConfig->getNoLlmDataReceivedMessage(), 'err');
|
yield $this->systemMsg($this->agentRunnerConfig->getNoLlmDataReceivedMessage(), 'err');
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
return $fullOutput;
|
return $fullOutput;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Build a deterministic safety answer for environments where the LLM returns no tokens.
|
||||||
|
*
|
||||||
|
* This intentionally does not infer technical suitability from weak evidence. It only reports
|
||||||
|
* reliable system state, shop hit metadata and the next safe action.
|
||||||
|
*
|
||||||
|
* @param string[] $knowledgeChunks
|
||||||
|
* @param ShopProductResult[] $shopResults
|
||||||
|
*/
|
||||||
|
private function buildNoLlmFallbackAnswer(
|
||||||
|
string $prompt,
|
||||||
|
string $urlContent,
|
||||||
|
array $knowledgeChunks,
|
||||||
|
array $shopResults,
|
||||||
|
string $commerceIntent,
|
||||||
|
bool $shopSearchAttempted,
|
||||||
|
bool $shopSearchHadSystemFailure,
|
||||||
|
?string $shopSearchFailureReason
|
||||||
|
): string {
|
||||||
|
$hasKnowledge = $knowledgeChunks !== [] || trim($urlContent) !== '';
|
||||||
|
$hasShopResults = $shopResults !== [];
|
||||||
|
$isCommerceIntent = $this->isCommerceIntent($commerceIntent);
|
||||||
|
|
||||||
|
if ($hasShopResults) {
|
||||||
|
return $this->buildNoLlmShopFallbackAnswer(
|
||||||
|
hasKnowledge: $hasKnowledge,
|
||||||
|
shopResults: $shopResults
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($shopSearchHadSystemFailure) {
|
||||||
|
return $this->buildNoLlmShopUnavailableAnswer(
|
||||||
|
hasKnowledge: $hasKnowledge,
|
||||||
|
reason: $shopSearchFailureReason
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($isCommerceIntent && $shopSearchAttempted) {
|
||||||
|
return $this->buildNoLlmNoShopResultsAnswer($hasKnowledge);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($hasKnowledge) {
|
||||||
|
return $this->agentRunnerConfig->getNoLlmFallbackKnowledgeOnlyMessage();
|
||||||
|
}
|
||||||
|
|
||||||
|
return $this->agentRunnerConfig->getNoLlmFallbackNoDataMessage();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param ShopProductResult[] $shopResults
|
||||||
|
*/
|
||||||
|
private function buildNoLlmShopFallbackAnswer(bool $hasKnowledge, array $shopResults): string
|
||||||
|
{
|
||||||
|
$intro = $hasKnowledge
|
||||||
|
? $this->agentRunnerConfig->getNoLlmFallbackShopWithKnowledgeMessage()
|
||||||
|
: $this->agentRunnerConfig->getNoLlmFallbackShopOnlyMessage();
|
||||||
|
|
||||||
|
$lines = [$intro, ''];
|
||||||
|
|
||||||
|
foreach ($this->buildNoLlmShopProductLines($shopResults) as $line) {
|
||||||
|
$lines[] = $line;
|
||||||
|
}
|
||||||
|
|
||||||
|
$escalation = trim($this->agentRunnerConfig->getNoLlmFallbackEscalationMessage());
|
||||||
|
if ($escalation !== '') {
|
||||||
|
$lines[] = '';
|
||||||
|
$lines[] = $escalation;
|
||||||
|
}
|
||||||
|
|
||||||
|
return trim(implode("\n", $lines));
|
||||||
|
}
|
||||||
|
|
||||||
|
private function buildNoLlmShopUnavailableAnswer(bool $hasKnowledge, ?string $reason): string
|
||||||
|
{
|
||||||
|
$reason = $this->normalizeOneLine((string) $reason);
|
||||||
|
$message = $hasKnowledge
|
||||||
|
? $this->agentRunnerConfig->getNoLlmFallbackShopUnavailableWithKnowledgeMessage()
|
||||||
|
: $this->agentRunnerConfig->getNoLlmFallbackShopUnavailableNoKnowledgeMessage();
|
||||||
|
|
||||||
|
if ($reason !== '') {
|
||||||
|
$message .= ' Ursache: ' . $reason;
|
||||||
|
}
|
||||||
|
|
||||||
|
return trim($message);
|
||||||
|
}
|
||||||
|
|
||||||
|
private function buildNoLlmNoShopResultsAnswer(bool $hasKnowledge): string
|
||||||
|
{
|
||||||
|
return $hasKnowledge
|
||||||
|
? $this->agentRunnerConfig->getNoLlmFallbackNoShopResultsWithKnowledgeMessage()
|
||||||
|
: $this->agentRunnerConfig->getNoLlmFallbackNoShopResultsNoKnowledgeMessage();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param ShopProductResult[] $shopResults
|
||||||
|
* @return string[]
|
||||||
|
*/
|
||||||
|
private function buildNoLlmShopProductLines(array $shopResults): array
|
||||||
|
{
|
||||||
|
$maxResults = max(1, $this->agentRunnerConfig->getNoLlmFallbackMaxShopResults());
|
||||||
|
$lines = [];
|
||||||
|
$index = 1;
|
||||||
|
|
||||||
|
foreach ($shopResults as $product) {
|
||||||
|
if (!$product instanceof ShopProductResult) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
$lines[] = $this->formatNoLlmShopProductLine($product, $index);
|
||||||
|
$index++;
|
||||||
|
|
||||||
|
if (count($lines) >= $maxResults) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($lines === []) {
|
||||||
|
return ['- Es wurden Shop-Treffer übergeben, aber keine lesbaren Produktdaten gefunden.'];
|
||||||
|
}
|
||||||
|
|
||||||
|
return $lines;
|
||||||
|
}
|
||||||
|
|
||||||
|
private function formatNoLlmShopProductLine(ShopProductResult $product, int $index): string
|
||||||
|
{
|
||||||
|
$parts = [];
|
||||||
|
|
||||||
|
$name = $this->normalizeOneLine($product->name);
|
||||||
|
$parts[] = $name !== '' ? $name : 'Unbenanntes Shop-Produkt';
|
||||||
|
|
||||||
|
if ($product->productNumber !== null && trim($product->productNumber) !== '') {
|
||||||
|
$parts[] = 'Art.-Nr. ' . $this->normalizeOneLine($product->productNumber);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($product->manufacturer !== null && trim($product->manufacturer) !== '') {
|
||||||
|
$parts[] = 'Hersteller: ' . $this->normalizeOneLine($product->manufacturer);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($product->price !== null && trim($product->price) !== '') {
|
||||||
|
$parts[] = 'Preis: ' . $this->normalizeOneLine($product->price);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($product->available !== null) {
|
||||||
|
$parts[] = 'Verfügbar: ' . ($product->available ? 'ja' : 'nein');
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($product->url !== null && trim($product->url) !== '') {
|
||||||
|
$parts[] = 'URL: ' . $this->normalizeOneLine($product->url);
|
||||||
|
}
|
||||||
|
|
||||||
|
return sprintf('%d. %s', $index, implode(' | ', $parts));
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @param string[] $sources
|
* @param string[] $sources
|
||||||
*/
|
*/
|
||||||
|
|||||||
@@ -40,15 +40,24 @@ final readonly class PromptBuilder
|
|||||||
array $knowledgeChunks,
|
array $knowledgeChunks,
|
||||||
array $shopResults = [],
|
array $shopResults = [],
|
||||||
?bool $fullContext = false,
|
?bool $fullContext = false,
|
||||||
?string $swagFullOutPut = ''
|
?string $swagFullOutPut = '',
|
||||||
|
bool $commerceSearchAttempted = false,
|
||||||
|
bool $shopSearchHadSystemFailure = false
|
||||||
): string {
|
): string {
|
||||||
$prompt = $this->normalizeBlockText($prompt);
|
$prompt = $this->normalizeBlockText($prompt);
|
||||||
$urlContent = $this->normalizeBlockText($urlContent);
|
$urlContent = $this->normalizeBlockText($urlContent);
|
||||||
$swagFullOutPut = $this->normalizeNullableBlockText($swagFullOutPut);
|
$swagFullOutPut = $this->normalizeNullableBlockText($swagFullOutPut);
|
||||||
|
|
||||||
$hasShopResults = $shopResults !== [];
|
$hasShopResults = $shopResults !== [];
|
||||||
|
$hasKnowledge = $knowledgeChunks !== [] || $urlContent !== '';
|
||||||
$isTechnicalProductQuestion = $this->isLikelyTechnicalProductQuestion($prompt);
|
$isTechnicalProductQuestion = $this->isLikelyTechnicalProductQuestion($prompt);
|
||||||
$asksForAccessoryOrBundle = $this->asksForAccessoryOrBundle($prompt);
|
$asksForAccessoryOrBundle = $this->asksForAccessoryOrBundle($prompt);
|
||||||
|
$reliabilityState = $this->resolveReliabilityState(
|
||||||
|
hasKnowledge: $hasKnowledge,
|
||||||
|
hasShopResults: $hasShopResults,
|
||||||
|
commerceSearchAttempted: $commerceSearchAttempted,
|
||||||
|
shopSearchHadSystemFailure: $shopSearchHadSystemFailure
|
||||||
|
);
|
||||||
|
|
||||||
$systemBlock = $this->buildSystemBlock();
|
$systemBlock = $this->buildSystemBlock();
|
||||||
$shopBlock = $this->buildShopBlock($shopResults, $swagFullOutPut);
|
$shopBlock = $this->buildShopBlock($shopResults, $swagFullOutPut);
|
||||||
@@ -56,6 +65,13 @@ final readonly class PromptBuilder
|
|||||||
hasShopResults: $hasShopResults,
|
hasShopResults: $hasShopResults,
|
||||||
isTechnicalProductQuestion: $isTechnicalProductQuestion
|
isTechnicalProductQuestion: $isTechnicalProductQuestion
|
||||||
);
|
);
|
||||||
|
$fallbackEscalationBlock = $this->buildFallbackEscalationBlock(
|
||||||
|
reliabilityState: $reliabilityState,
|
||||||
|
hasShopResults: $hasShopResults,
|
||||||
|
commerceSearchAttempted: $commerceSearchAttempted,
|
||||||
|
shopSearchHadSystemFailure: $shopSearchHadSystemFailure,
|
||||||
|
isTechnicalProductQuestion: $isTechnicalProductQuestion
|
||||||
|
);
|
||||||
$responseFormatBlock = $this->buildResponseFormatBlock(
|
$responseFormatBlock = $this->buildResponseFormatBlock(
|
||||||
hasShopResults: $hasShopResults,
|
hasShopResults: $hasShopResults,
|
||||||
isTechnicalProductQuestion: $isTechnicalProductQuestion,
|
isTechnicalProductQuestion: $isTechnicalProductQuestion,
|
||||||
@@ -73,6 +89,7 @@ final readonly class PromptBuilder
|
|||||||
$systemBlock,
|
$systemBlock,
|
||||||
$shopBlock,
|
$shopBlock,
|
||||||
$outputPriorityBlock,
|
$outputPriorityBlock,
|
||||||
|
$fallbackEscalationBlock,
|
||||||
$responseFormatBlock,
|
$responseFormatBlock,
|
||||||
$knowledgeBlock,
|
$knowledgeBlock,
|
||||||
$userBlock,
|
$userBlock,
|
||||||
@@ -88,6 +105,7 @@ final readonly class PromptBuilder
|
|||||||
$systemBlock,
|
$systemBlock,
|
||||||
$shopBlock,
|
$shopBlock,
|
||||||
$outputPriorityBlock,
|
$outputPriorityBlock,
|
||||||
|
$fallbackEscalationBlock,
|
||||||
$responseFormatBlock,
|
$responseFormatBlock,
|
||||||
$knowledgeBlock,
|
$knowledgeBlock,
|
||||||
$contextBlock,
|
$contextBlock,
|
||||||
@@ -239,6 +257,66 @@ final readonly class PromptBuilder
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private function resolveReliabilityState(
|
||||||
|
bool $hasKnowledge,
|
||||||
|
bool $hasShopResults,
|
||||||
|
bool $commerceSearchAttempted,
|
||||||
|
bool $shopSearchHadSystemFailure
|
||||||
|
): string {
|
||||||
|
if ($shopSearchHadSystemFailure && !$hasKnowledge) {
|
||||||
|
return 'shopdaten_nicht_verfuegbar';
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($hasKnowledge && !$hasShopResults) {
|
||||||
|
return 'sicher_beantwortbar';
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($hasKnowledge && $hasShopResults) {
|
||||||
|
return 'wahrscheinlich_beantwortbar';
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!$hasKnowledge && $hasShopResults) {
|
||||||
|
return 'nur_shop_treffer_kein_belastbares_fachwissen';
|
||||||
|
}
|
||||||
|
|
||||||
|
return 'keine_belastbaren_daten';
|
||||||
|
}
|
||||||
|
|
||||||
|
private function buildFallbackEscalationBlock(
|
||||||
|
string $reliabilityState,
|
||||||
|
bool $hasShopResults,
|
||||||
|
bool $commerceSearchAttempted,
|
||||||
|
bool $shopSearchHadSystemFailure,
|
||||||
|
bool $isTechnicalProductQuestion
|
||||||
|
): string {
|
||||||
|
$rules = [];
|
||||||
|
$stateLineTemplate = $this->config->getFallbackEscalationStateLineTemplate();
|
||||||
|
|
||||||
|
if ($stateLineTemplate !== '') {
|
||||||
|
$rules[] = str_replace('{state}', $reliabilityState, $stateLineTemplate);
|
||||||
|
}
|
||||||
|
|
||||||
|
$rules = array_merge($rules, $this->config->getFallbackEscalationBaseRules());
|
||||||
|
$rules = array_merge($rules, $this->config->getFallbackEscalationStateRules($reliabilityState));
|
||||||
|
|
||||||
|
if ($isTechnicalProductQuestion && !$commerceSearchAttempted && !$shopSearchHadSystemFailure) {
|
||||||
|
$rules = array_merge($rules, $this->config->getFallbackEscalationWithoutShopCheckRules());
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($hasShopResults && !$commerceSearchAttempted) {
|
||||||
|
$rules[] = '- Treat shop results as provided context only; do not imply that a live shop check was performed in this run.';
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($rules === []) {
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
|
||||||
|
return $this->buildRuleBlock(
|
||||||
|
$this->config->getFallbackEscalationSectionLabel(),
|
||||||
|
$rules
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
private function buildResponseFormatBlock(
|
private function buildResponseFormatBlock(
|
||||||
bool $hasShopResults,
|
bool $hasShopResults,
|
||||||
bool $isTechnicalProductQuestion,
|
bool $isTechnicalProductQuestion,
|
||||||
|
|||||||
@@ -149,7 +149,7 @@ final class AgentRunnerConfig
|
|||||||
{
|
{
|
||||||
return $this->getString(
|
return $this->getString(
|
||||||
'messages.no_concrete_shop_query',
|
'messages.no_concrete_shop_query',
|
||||||
'Ich habe keine konkrete Shop-Suchanfrage erkannt. Bitte nenne das Produkt, Zubehör oder die Artikelnummer.'
|
'Ich kann die Shop-Suche noch nicht belastbar auflösen. Bitte nenne das Produkt, den Messparameter oder das Zubehör etwas konkreter.'
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -173,6 +173,83 @@ final class AgentRunnerConfig
|
|||||||
return $this->getString('messages.no_llm_data_received', '❌ Es wurden keine Daten vom LLM empfangen.');
|
return $this->getString('messages.no_llm_data_received', '❌ Es wurden keine Daten vom LLM empfangen.');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackMaxShopResults(): int
|
||||||
|
{
|
||||||
|
return $this->getInt('no_llm_fallback.max_shop_results', 5);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackShopOnlyMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.shop_only',
|
||||||
|
'Ich finde dazu im RAG-Wissen keine belastbare Fachinformation. Aus den Shopdaten ergeben sich folgende Treffer; technische Eignung bitte prüfen:'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackShopWithKnowledgeMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.shop_with_knowledge',
|
||||||
|
'Es liegen RAG-/Kontexttreffer und Shopdaten vor. Ohne LLM leite ich daraus keine technische Eignung ab. Die Shopdaten zeigen folgende Treffer; technische Eignung bitte prüfen:'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackEscalationMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.escalation',
|
||||||
|
'Für eine verbindliche Produktauswahl sollte der konkrete Anwendungsfall durch Vertrieb oder Support geprüft werden.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackKnowledgeOnlyMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.knowledge_only',
|
||||||
|
'Ich habe Treffer im RAG-Wissen gefunden, aber ohne LLM kann ich daraus keine belastbare fachliche Antwort synthetisieren. Ich gebe deshalb keine sichere Produktaussage aus. Bitte aktiviere das LLM oder konkretisiere die Frage für eine gezielte Prüfung.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackNoDataMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.no_data',
|
||||||
|
'Ich finde dazu keine belastbaren Daten in den vorliegenden Quellen. Bitte nenne Produkt, Messparameter, Zubehör oder Anwendungsfall genauer.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackNoShopResultsWithKnowledgeMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.no_shop_results_with_knowledge',
|
||||||
|
'Ich finde RAG-/Kontexttreffer, aber keine passenden Shop-Treffer zur aktuellen Suchanfrage. Das ist keine Aussage, dass es das Produkt nicht gibt. Ohne LLM gebe ich keine technische Negativaussage aus; bitte prüfe den Suchbegriff oder den Anwendungsfall gezielter.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackNoShopResultsNoKnowledgeMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.no_shop_results_no_knowledge',
|
||||||
|
'Ich finde weder belastbares RAG-Wissen noch passende Shop-Treffer zur aktuellen Suchanfrage. Das ist keine sichere Negativaussage. Bitte nenne Produkt, Messparameter oder Zubehör konkreter.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackShopUnavailableWithKnowledgeMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.shop_unavailable_with_knowledge',
|
||||||
|
'Live-Shopdaten konnten nicht geladen werden. Ohne Shop-Check treffe ich keine Aussage zu aktueller Verfügbarkeit, Preis oder Shop-Portfolio. Vorhandenes RAG-Wissen darf nur als fachlicher Kontext verstanden werden.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getNoLlmFallbackShopUnavailableNoKnowledgeMessage(): string
|
||||||
|
{
|
||||||
|
return $this->getString(
|
||||||
|
'no_llm_fallback.messages.shop_unavailable_no_knowledge',
|
||||||
|
'Live-Shopdaten konnten nicht geladen werden und ich finde kein belastbares RAG-Wissen. Ich kann daraus keine verlässliche Produkt- oder Verfügbarkeitsaussage ableiten.'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
public function getGenericInternalErrorMessage(): string
|
public function getGenericInternalErrorMessage(): string
|
||||||
{
|
{
|
||||||
return $this->getString('messages.generic_internal_error', '❌ Bei der Verarbeitung der Anfrage ist ein interner Fehler aufgetreten.');
|
return $this->getString('messages.generic_internal_error', '❌ Bei der Verarbeitung der Anfrage ist ein interner Fehler aufgetreten.');
|
||||||
|
|||||||
@@ -310,6 +310,50 @@ final class PromptBuilderConfig
|
|||||||
]);
|
]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public function getFallbackEscalationSectionLabel(): string
|
||||||
|
{
|
||||||
|
return $this->getString('sections.fallback_escalation_label', 'FALLBACK AND ESCALATION RULES');
|
||||||
|
}
|
||||||
|
|
||||||
|
public function getFallbackEscalationStateLineTemplate(): string
|
||||||
|
{
|
||||||
|
return $this->getString('fallback_escalation.state_line_template', '- Internal confidence state: {state}.');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @return string[]
|
||||||
|
*/
|
||||||
|
public function getFallbackEscalationBaseRules(): array
|
||||||
|
{
|
||||||
|
return $this->getStringList('fallback_escalation.base_rules', [
|
||||||
|
'- Prefer transparent uncertainty over a confident but unsupported answer.',
|
||||||
|
'- Never present missing or weak evidence as proof that a product, value, accessory, or suitability does not exist.',
|
||||||
|
'- A negative answer is allowed only when the provided sources explicitly support that negative finding for the asked scope.',
|
||||||
|
'- If several products, parameters, or accessories could match, ask one focused clarification question instead of guessing.',
|
||||||
|
'- For risky or binding product selection, state that sales or support should verify the application before a final selection.',
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @return string[]
|
||||||
|
*/
|
||||||
|
public function getFallbackEscalationStateRules(string $state): array
|
||||||
|
{
|
||||||
|
return $this->getStringList('fallback_escalation.states.' . $state, []);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @return string[]
|
||||||
|
*/
|
||||||
|
public function getFallbackEscalationWithoutShopCheckRules(): array
|
||||||
|
{
|
||||||
|
return $this->getStringList('fallback_escalation.without_shop_check_rules', [
|
||||||
|
'- If the question is product-related and no live shop check was performed in this run, do not make a portfolio-wide negative statement such as "there is no product".',
|
||||||
|
'- Phrase missing evidence narrowly, for example: "Im RAG-Wissen finde ich dazu keine belastbare Information."',
|
||||||
|
'- If useful, say that a shop search can be used to look for matching products, but do not claim shop results were checked unless they are present in the prompt.',
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
public function getResponseFormatSectionLabel(): string
|
public function getResponseFormatSectionLabel(): string
|
||||||
{
|
{
|
||||||
return $this->getString('sections.response_format_label', 'RESPONSE FORMAT RULES');
|
return $this->getString('sections.response_format_label', 'RESPONSE FORMAT RULES');
|
||||||
|
|||||||
Reference in New Issue
Block a user