Reliability, Escalation ve 76-Soru Tam Simülasyon

Reliability, agent’ın tekrarlanabilir şekilde doğru cevap üretmesi demektir. Bu derste reliability’yi üç başlıkta işliyoruz: (1) escalation — ne zaman insan devreye girmeli, (2) error propagation — multi-agent sistemde hata nasıl yayılır ve nasıl kontrol edilir, (3) calibration — model ne zaman “bilmiyorum” demeli; rehber ayrıca provenance/attribution’ı (Ch 12) da reliability’nin parçası sayar — onu Ders 10-11’in provenance bölümlerinde işledik. Bu dersin ikinci yarısı, resmî hazırlık rehberindeki practice testin tamamını kapsayan 76 soruluk tam bir pratik sınav (gerçek sınav 60 sorudur; buradaki 76 soru 5 senaryonun tüm havuzunu kapsar). Domain dağılımı sınavdaki ağırlıkları yansıtır; her sorunun gerekçesiyle birlikte çalış, sınavdan önce bu dosyayı bitir.

Neden bu konu sınavda çıkıyor

Reliability, sertifika sınavının kırılma noktası. Çoğu aday agent mimarisini, tool tasarımını, prompt mühendisliğini bilir — ama “agent ne zaman escalate etmeli” veya “multi-agent hata yayılımını nasıl önlerim” sorularında tökezler. Sınav senaryolarının çoğu reliability’yi test eder: “Müşteri 3 kez aynı soruyu sordu, ne yaparsın?” gibi. Domain 5’in (Context Management + Reliability) %15’i bu derste.

Temel kavramlar

Üç escalation tetikleyicisi

Escalation otomatik değil; deterministik tetikleyici lazım. Üç kategori:

1. Açık insan talebi. Kullanıcı doğrudan “insana bağla” veya “yetkiliye yönlendir” dediğinde. Bu non-negotiable — sentiment analizi ile tahmin yürütme, kelimeyi duyduğun an escalation.

2. Policy gap. Agent’ın policy/referans dökümanında net bir cevap yok. Örneğin “50.000$ üzeri iade için yönetici onayı şart” policy’si varsa ve case 60.000$, otomatik escalation. Deterministik: kuralı kontrol et, uyuyorsa escalate.

3. İlerleme yokluğu (loop detection — rehberin ifadesiyle “the agent cannot make progress → escalate after a reasonable number of attempts”). Aynı tool call ardarda 2-3 kez başarısız oluyorsa veya agent aynı soruya 3 kez aynı cevabı veriyorsa → escalation. Programmatic: tool call count tracker, response similarity check.

Sentiment / self-confidence güvenilmez

Sınav tuzaklarından biri: “agent’ın self-rated confidence’ına bakarak escalate et” veya “müşteri kızgın görünüyor, escalate et” → ikisi de yanlış.

Self-rated confidence: LLM’in “I’m 90% confident” demesi calibration yapmaz. Pratikte %90 dediği cevap %75 doğrudur. Aggregate metrikler de bunu maskeler. Nüans: self-rated confidence final karar için güvenilmezdir; ama review routing için kullanılabilir (Domain 4.6: verification pass’leri self-rated confidence ile kalibre ederek yönlendir). Yani yönlendirme sinyali evet, otomatik onay hayır.
Sentiment analysis: “Angry customer” sinyali, escalation tetikleyicisi değildir. Kızgın ama basit bir soru → agent çözebilir. Sakin ama policy gap’li bir case → escalate gerek. İçerik tetikler, duygu değil.

Doğru yaklaşım: escalation kararını deterministik kurallar ve programmatic precondition’lar üzerine kur. Duygu/özgüven sinyali bilgi olarak işe yarar ama karar verici olamaz.

Multi-agent hata yayılımı

Hub-and-spoke mimarisinde coordinator subagent çağırır, sonuç döner, coordinator devam eder. Hata burada iki şekilde büyür:

(a) Tool call hatasının yutulması. Subagent tool call başarısız olduğunda sessizce boş array döndürürse, coordinator “veri yok” sanıp yanlış sonuç üretebilir. Çözüm: structured error context — her subagent dönüşünde şu şema:

{
  "status": "partial | success | failed",
  "failure_type": "transient | validation | business | permission",
  "query": "get_orders(customer_id=CUST-77821, status=open)",
  "partial_results": [...],
  "alternatives_considered": ["tried with status=open", "tried with no status filter"],
  "next_step_recommendation": "escalate_to_human | retry_with_backoff | try_alternative"
}

(b) Access failure vs valid empty result. API 404 döndü → kayıt yok mu, yoksa erişim mi yok? Bu ayrım kritik. Çözüm: status code + reason mapping — 404 + “record not found” ≠ 403 + “permission denied”. Agent’ın her ikisini de görmesi gerekir. 📝 Rehberin kuralı “access failure vs valid empty result” ayrımıdır; status code + reason mapping bunun pratik uygulamasıdır.

Attribution loss problemi

Multi-agent sistemde subagent A “X doğrudur” dedi, subagent B “Y doğrudur” dedi, coordinator sentezledi. Sonuçta hangi bilgi nereden geldi kaybolur. Buna attribution loss denir. Çözüm: yapısal mapping koruma — her claim’in kaynağı subagent_id + tool_call_id + retrieval_doc_id ile birlikte tutulur:

{
  "claim": "Müşteri 3 kez iade talep etti",
  "source": {
    "subagent_id": "support-history",
    "tool_call_id": "get_refund_history:77821",
    "doc_id": "refund-2024-11-03"
  },
  "confidence": 0.92
}

Sınavda bu “claim→source bağını koparma” anti-pattern’i sıkça test edilir. Doğru: annotation’lar korunur, sentez değil. 📝 Render by content type kuralını unutma: finansal veri → tablo, haber → düz metin, teknik bulgu → yapılandırılmış liste (Ch 12.4).

Çelişen istatistikler

İki farklı kaynak farklı sayı veriyorsa (ör. “Toplam kullanıcı: 1,247” vs “1,250”), agent birini seçmemeli. Annotation ile sakla:

{
  "metric": "total_users",
  "values": [
    {"source": "analytics_db", "value": 1247, "as_of": "2024-11-10"},
    {"source": "billing_db", "value": 1250, "as_of": "2024-11-09"}
  ],
  "resolution": "report both with timestamps; do not select one arbitrarily"
}

Temporal fark yorumlama: “2024-11-09” ve “2024-11-10” snapshot’ları arasındaki 3 fark yeni signup olabilir. Agent yorumlamalı, keyfi seçmemeli.

Coverage annotation

Hangi agent hangi alanı kontrol etti, hangi alanı atladı? Coverage annotation:

{
  "fields_checked": {
    "refund_eligibility": {"agent": "policy-agent", "result": "eligible", "confidence": 0.95},
    "duplicate_check":    {"agent": "history-agent", "result": "no_duplicate", "confidence": 0.88},
    "manager_approval":   {"agent": "policy-agent", "result": "not_required", "confidence": 0.92},
    "fraud_signal":       {"agent": null, "result": null, "note": "out of scope, escalate"}
  }
}

Sınavda “complete coverage” ile “selective coverage” arasındaki fark sorulur. Selective coverage = incomplete (eksik alanları belirt, kör nokta bırakma).

Stratified random sampling + field-level confidence

Calibration için iki prensip:

Stratified random sampling: Veriyi katmanlara böl (case tipine, müşteri segmentine, ürün kategorisine göre) ve her katmandan rastgele örnekle. Aggregate “ortalama doğruluk %92” yerine katman bazında: “VIP case’lerinde %97, standart case’lerde %89, dispute case’lerinde %78”. Aggregate hata maskeler.
Field-level confidence: Her alan için ayrı confidence skor. Genel “I’m confident” yerine: refund_amount: 0.95, customer_status: 0.88, delivery_date: 0.72. Field-level darboğazı görünür kılar.

Yüksek false-positive kategorisini geçici disable et. Bir review/reliability kategorisi (örn. “performance”) sürekli yanlış alarm üretiyorsa en hızlı güven-koruma hamlesi o kategoriyi geçici olarak kapatmak, prompt’u düzeltince geri açmaktır. Few-shot eklemek, confidence göstermek veya genel strictness kısmak kök nedeni çözmez. Sınav bu kalıbı birden fazla soruda tekrarlar.

Sınav tuzakları

“Sentiment analysis ile escalation tetikle” → YANLIŞ. İçerik tetikler, duygu değil.
“Self-rated confidence’a güven” → YANLIŞ. Calibration yapmaz; programmatic verification gerek.
“Subagent tool call hatasını yut” → YANLIŞ. Structured error context zorunlu.
“404 döndüyse kayıt yoktur” → YANLIŞ. 404 ≠ permission denied. Status code + reason mapping. 📝 Rehberin kuralı “access failure vs valid empty result” ayrımıdır; status code + reason mapping bunun pratik uygulamasıdır.
“İki kaynak farklı sayı veriyorsa birini seç” → YANLIŞ. İkisini de annotation ile sakla, temporal fark yorumla.
“Coverage annotation’ı sentezle ve göm” → YANLIŞ. Selective coverage = incomplete; kör noktayı açık bırak.
“Aggregate %95 confidence yeterli” → YANLIŞ. Field-level + stratified sampling gerek.
“Sürekli çalışan agent loop güvenilirdir” → YANLIŞ. Loop detection + escalation tetikleyicisi şart.

Hands-on

// Programmatic escalation trigger örneği
function shouldEscalate(context: AgentContext): EscalationDecision {
  if (context.user_explicit_request) {
    return { escalate: true, reason: "explicit_human_request" };
  }
  if (context.policy_gap_detected) {
    return { escalate: true, reason: "policy_gap" };
  }
  if (context.repeated_tool_failures >= 2 || context.repeated_responses >= 3) {
    return { escalate: true, reason: "no_progress" };
  }
  return { escalate: false };
}

// Structured error context döngüsü
async function safeSubagentCall(task: Task): Promise<SubagentResult> {
  try {
    const result = await Task({ ...task });
    return { status: "success", data: result, partial: false };
  } catch (err) {
    return {
      status: "failed",
      failure_type: classifyError(err),  // transient | validation | business | permission
      query: task.prompt,
      partial_results: err.partial || [],
      next_step: err.transient ? "retry_with_backoff" : "escalate_to_human",
    };
  }
}

Özet

Escalation 3 tetikleyici: açık insan talebi, policy gap, ilerleme yokluğu. Duygu/güven sinyali karar verici değil.
Multi-agent hata yayılımı structured error context ile kontrol edilir; silent failure en tehlikeli pattern.
Access failure (403) vs valid empty (404) ayrımı status code + reason mapping ile yapılır.
Attribution loss = claim→source bağı kopması; yapısal mapping her zaman korunur.
Çelişen istatistikler keyfi seçilmez; annotation + temporal fark yorumu ile saklanır.
Coverage selective ise incomplete olarak işaretlenir; kör nokta gizlenmez.
Aggregate metrikler hata maskeler; stratified sampling + field-level confidence ile calibration yapılır.

76-Soru Tam Simülasyon

Aşağıdaki 76 soru, resmî hazırlık rehberindeki practice testin tamamını kapsar: 4 çekirdek senaryo (Multi-Agent Research, CI/CD, Code Generation, Customer Support) her biri 15 soru + Conversational AI Architecture Patterns senaryosu 16 soru. Gerçek sınav 60 sorudur; bu simülasyonun tamamını çözen aday tüm senaryo havuzunu görmüş olur. Sınav formatına tam uyum amacıyla durumlar, sorular ve seçenekler orijinal İngilizce dilinde tutulmuştur; her sorunun altında Türkçe detaylı gerekçesi yer almaktadır.

Q1: Multi-agent Research System (Scenario 1)

Situation: A document analysis agent discovers that two credible sources contain directly contradictory statistics for a key metric: a government report states 40% growth, while an industry analysis states 12%. Both sources look credible, and the discrepancy could materially affect the research conclusions. How should the document analysis agent handle this situation most effectively?

Question: Which approach is most effective?

Options:

A) Apply credibility heuristics to pick the most likely correct number, finish analysis with that value, and add a footnote mentioning the discrepancy. (Correct: False)
B) Include both numbers in the analysis output without marking them as conflicting, letting the synthesis agent decide which to use based on broader context. (Correct: False)
C) Stop analysis and immediately escalate to the coordinator, asking it to decide which source is more authoritative before continuing. (Correct: False)
D) Complete analysis with both numbers, explicitly annotate the conflict with source attribution, and let the coordinator decide how to reconcile the data before passing to synthesis. (Correct: True)

Correct Answer: D

Explanation (Gerekçe): Bu yaklaşım sorumlulukların ayrılması (separation of responsibilities) prensibini korur: analiz agent’ı bloklanmadan ana görevini tamamlar, çelişen her iki değeri de net kaynak atıflarıyla muhafaza eder ve uzlaştırma (reconciliation) kararını, daha geniş bir bağlama (context) sahip olan coordinator’a doğru bir şekilde devreder.

Q2: Multi-agent Research System (Scenario 2)

Situation: The web-search and document-analysis agents have completed their tasks and returned results to the coordinator. What is the next step for creating an integrated research report?

Question: Which next step is most appropriate?

Options:

A) Each agent sends its results directly to the report-writing agent, bypassing the coordinator. (Correct: False)
B) The document analysis agent requests web-search results and merges them internally. (Correct: False)
C) The coordinator passes both sets of results to the synthesis agent for a unified integration. (Correct: True)
D) The coordinator concatenates the raw outputs from both agents and returns them as the final result. (Correct: False)

Correct Answer: C

Explanation (Gerekçe): Bir coordinator–subagent mimarisinde, coordinator, kontrolü elde tutmak ve yüksek kaliteli bir birleştirmeyi (merging) garanti etmek adına her iki sonuç kümesini de merkezi entegrasyon için synthesis agent’ına iletir.

Q3: Multi-agent Research System (Scenario 3)

Situation: A document analysis subagent frequently fails when processing PDF files: some have corrupted sections that trigger parsing exceptions, others are password-protected, and sometimes the parsing library hangs on large files. Currently, any exception immediately terminates the subagent and returns an error to the coordinator, which must decide whether to retry, skip, or fail the whole task. This causes excessive coordinator involvement in routine error handling. What architectural improvement is most effective?

Question: Which improvement is most effective?

Options:

A) Create a dedicated error-handling agent that monitors all failures via a shared queue and decides recovery actions, sending restart commands directly to subagents. (Correct: False)
B) Configure the subagent to always return partial results with a success status, embedding error details in metadata; the coordinator treats all responses as successful. (Correct: False)
C) Make the coordinator validate all documents before sending them to the subagent, rejecting documents that might cause failures. (Correct: False)
D) Implement local recovery in the subagent for transient failures and escalate to the coordinator only errors it cannot resolve, including attempted steps and partial results. (Correct: True)

Correct Answer: D

Explanation (Gerekçe): Hataları, bunları çözebilecek en alt seviyede ele alın. Yerel kurtarma (local recovery), coordinator’ın iş yükünü azaltırken, gerçekten kurtarılamaz olan sorunları tüm bağlam (context) ve kısmi ilerleme (partial progress) bilgileriyle birlikte coordinator’a iletmeye (escalate etmeye) devam eder.

Q4: Multi-agent Research System (Scenario 4)

Situation: After running the system on “AI impact on creative industries,” you observe that every subagent completes successfully: the web-search agent finds relevant articles, the document analysis agent summarizes them correctly, and the synthesis agent produces coherent text. However, final reports cover only visual art and completely miss music, literature, and film. In the coordinator logs, you see it decomposed the topic into three subtasks: “AI in digital art,” “AI in graphic design,” and “AI in photography.” What is the most likely root cause?

Question: What is the most likely root cause?

Options:

A) The synthesis agent lacks instructions to detect coverage gaps. (Correct: False)
B) The document analysis agent filters out non-visual sources due to overly strict relevance criteria. (Correct: False)
C) The coordinator’s task decomposition is too narrow, assigning subagents work that does not cover all relevant areas. (Correct: True)
D) The web-search agent’s queries are insufficient and should be broadened to cover more sectors. (Correct: False)

Correct Answer: C

Explanation (Gerekçe): coordinator geniş bir konuyu yalnızca görsel sanat (visual-art) alt görevlerine ayırmış; müzik, edebiyat ve sinemayı tamamen gözden kaçırmıştır. subagent’lar kendilerine verilen görevleri doğru bir şekilde yürüttükleri için, kök nedenin dar kapsamlı görev ayrımı (task decomposition) olduğu açıkça görülmektedir.

Q5: Multi-agent Research System (Scenario 5)

Situation: The web-search subagent returns results for only 3 of 5 requested source categories (competitor sites and industry reports succeed, but news archives and social feeds time out). The document analysis subagent successfully processes all provided documents. The synthesis subagent must produce a summary from mixed-quality upstream inputs. Which error-propagation strategy is most effective?

Question: Which error-propagation strategy is most effective?

Options:

A) Continue synthesis using only successful sources and produce an output without mentioning which data was unavailable. (Correct: False)
B) The synthesis subagent returns an error to the coordinator, triggering a full retry or task failure due to incomplete data. (Correct: False)
C) The synthesis subagent asks the coordinator to retry timed-out sources with a longer timeout before starting synthesis. (Correct: False)
D) Structure the synthesis output with coverage annotations that indicate which conclusions are well-supported and where gaps exist due to unavailable sources. (Correct: True)

Correct Answer: D

Explanation (Gerekçe): Kapsam açıklamaları (coverage annotations), şeffaflıkla birlikte aşamalı kesintiyi (graceful degradation) uygular; tamamlanan işlerden elde edilen değeri korurken, güvenilirlik derecesine yönelik bilinçli kararlar verilmesini sağlamak amacıyla belirsizliği üst katmanlara iletir.

Q6: Multi-agent Research System (Scenario 6)

Situation: The document analysis subagent encounters a corrupted PDF file that it cannot parse. When designing the system’s error handling, what is the most effective way to handle this failure?

Question: Which approach is most effective?

Options:

A) Return an error with context to the coordinator agent, allowing it to decide how to proceed. (Correct: True)
B) Silently skip the corrupted document and continue processing the remaining files to avoid interrupting the workflow. (Correct: False)
C) Automatically retry parsing the document three times with exponential backoff before reporting a failure. (Correct: False)
D) Throw an exception that terminates the entire research workflow. (Correct: False)

Correct Answer: A

Explanation (Gerekçe): coordinator agent’ına bağlam bilgisi içeren (with context) bir hata döndürmek en etkili yaklaşımdır. Çünkü bu yaklaşım, coordinator’ın bilinçli bir karar vermesini sağlarken (dosyayı atlamak, alternatif bir ayrıştırma yöntemi denemek veya kullanıcıyı bilgilendirmek gibi) hatanın görünürlüğünü de korur.

Q7: Multi-agent Research System (Scenario 7)

Situation: Production logs show a persistent pattern: requests like “analyze the uploaded quarterly report” are routed to the web-search agent 45% of the time instead of the document analysis agent. Reviewing tool definitions, you find that the web-search agent has a tool analyze_content described as “analyzes content and extracts key information,” while the document analysis agent has a tool analyze_document described as “analyzes documents and extracts key information.” How should you fix the misrouting problem?

Question: How should you fix the misrouting problem?

Options:

A) Add a pre-routing classifier that detects whether the user refers to uploaded files or web content before the coordinator decides on delegation. (Correct: False)
B) Rename the web-search tool to extract_web_results and update its description to “processes and returns information retrieved from web search and URLs.” (Correct: True)
C) Add few-shot examples to the coordinator prompt showing correct routing: “User uploads a quarterly report → document analysis agent” and “User asks about a web page → web-search agent.” (Correct: False)
D) Expand the document analysis tool description with usage examples like “Use for uploaded PDFs, Word docs, and spreadsheets,” leaving the web-search tool unchanged. (Correct: False)

Correct Answer: B

Explanation (Gerekçe): Web-search aracını extract_web_results olarak yeniden adlandırmak ve açıklamasını web araması ve URL’lere açıkça atıfta bulunacak şekilde güncellemek, iki araç ismi ve açıklaması arasındaki anlamsal örtüşmeyi (semantic overlap) ortadan kaldırarak kök nedeni doğrudan çözer. Bu, her bir aracın amacını netleştirerek coordinator’ın document analysis ile web-search işlemlerini güvenilir bir şekilde ayırt etmesini sağlar.

Q8: Multi-agent Research System (Scenario 8)

Situation: A colleague proposes that the document analysis agent should send its results directly to the synthesis agent, bypassing the coordinator. What is the main advantage of keeping the coordinator as the central hub for all communication between subagents?

Question: What is the main advantage of keeping the coordinator as the central hub?

Options:

A) The coordinator can observe all interactions, handle errors uniformly, and decide what information each subagent should receive. (Correct: True)
B) The coordinator batches multiple requests to subagents, reducing total API calls and overall latency. (Correct: False)
C) Routing through the coordinator enables automatic retry logic that direct inter-agent calls cannot support. (Correct: False)
D) Subagents use isolated memory, and direct communication would require complex serialization that only the coordinator can perform. (Correct: False)

Correct Answer: A

Explanation (Gerekçe): coordinator deseni (coordinator pattern), tüm etkileşimler üzerinde merkezi bir görünürlük (centralized visibility), sistem genelinde tek biçimli hata yönetimi (uniform error handling) ve her bir subagent’ın hangi bilgiyi alacağı üzerinde hassas kontrol (fine-grained control) sağlar; bunlar, yıldız şeklindeki (star-shaped) bir iletişim topolojisinin birincil avantajlarıdır.

Q9: Multi-agent Research System (Scenario 9)

Situation: The web-search subagent times out while researching a complex topic. You need to design how information about this failure is returned to the coordinator. Which error-propagation approach best enables intelligent recovery?

Question: Which error-propagation approach best enables intelligent recovery?

Options:

A) Return structured error context to the coordinator including the failure type, the query executed, any partial results, and potential alternative approaches. (Correct: True)
B) Catch the timeout within the subagent and return an empty result set marked as successful. (Correct: False)
C) Implement automatic exponential-backoff retries inside the subagent, only returning a generic “search unavailable” status after exhausting retries. (Correct: False)
D) Propagate the timeout exception directly to the top-level handler, terminating the entire research workflow. (Correct: False)

Correct Answer: A

Explanation (Gerekçe): Hata türü, yürütülen sorgu (query), kısmi sonuçlar ve potansiyel alternatif yaklaşımlar dahil olmak üzere yapılandırılmış hata bağlamının (structured error context) coordinator’a döndürülmesi; coordinator’a akıllı kurtarma kararları (örneğin, düzenlenmiş bir sorguyla yeniden denemek veya kısmi sonuçlarla devam etmek gibi) vermesi için gereken her şeyi sağlar. Bu yaklaşım, coordinator seviyesinde bilinçli kararlar alınabilmesi için maksimum bağlamı (context) korur.

Q10: Multi-agent Research System (Scenario 10)

Situation: In your system design, you gave the document analysis agent access to a general-purpose tool fetch_url so it could download documents by URL. Production logs show this agent now frequently downloads search engine results pages to perform ad hoc web search—behavior that should be routed through the web-search agent—causing inconsistent results. Which fix is most effective?

Question: Which fix is most effective?

Options:

A) Replace fetch_url with a load_document tool that validates that URLs point to document formats. (Correct: True)
B) Remove fetch_url from the document analysis agent and route all URL fetching through the coordinator to the web-search agent. (Correct: False)
C) Implement filtering that blocks fetch_url calls to known search engine domains while allowing other URLs. (Correct: False)
D) Add instructions to the document analysis agent prompt that fetch_url should only be used to download document URLs, not to search. (Correct: False)

Correct Answer: A

Explanation (Gerekçe): Genel amaçlı bir aracın, URL’lerin belge formatlarına uygunluğunu doğrulayan belgeye özel bir araçla (load_document) değiştirilmesi, yetenekleri arayüz düzeyinde kısıtlayarak kök nedeni çözer. Bu, en az yetki prensibini (principle of least privilege) izler ve istenmeyen arama davranışını sadece caydırmakla kalmayıp tamamen imkansız hale getirir.

Q11: Multi-agent Research System (Scenario 11)

Situation: While researching a broad topic, you observe that the web-search agent and the document analysis agent investigate the same subtopics, leading to substantial duplication in their outputs. Token usage nearly doubles without a proportional increase in research breadth or depth. What is the most effective way to address this?

Question: What is the most effective way to address this?

Options:

A) Allow both agents to finish in parallel, then have the coordinator deduplicate overlapping results before passing them to the synthesis agent. (Correct: False)
B) The coordinator explicitly partitions the research space before delegating, assigning each agent distinct subtopics or source types. (Correct: True)
C) Implement a shared-state mechanism where agents log their current focus area so other agents can dynamically avoid duplication during execution. (Correct: False)
D) Switch to sequential execution where document analysis runs only after web search completes, using web-search results as context to avoid duplication. (Correct: False)

Correct Answer: B

Explanation (Gerekçe): coordinator’ın görevlendirme yapmadan önce araştırma alanını açıkça bölümlere ayırması (partitioning the research space), çalışma başlamadan önce kök nedeni (belirsiz görev sınırları) doğrudan çözdüğü için en etkili yöntemdir. Bu yaklaşım, çakışan eforları ve boşa harcanan token’ları engellerken paralelliği de korur.

Q12: Multi-agent Research System (Scenario 12)

Situation: During research, the web-search subagent queries three source categories with different outcomes: academic databases return 15 relevant papers, industry reports return “0 results,” and patent databases return “Connection timeout.” When designing error propagation to the coordinator, which approach enables the best recovery decisions?

Question: Which approach enables the best recovery decisions?

Options:

A) Aggregate the results into a single success-percentage metric (e.g., “67% source coverage”) with detailed logs available on demand. (Correct: False)
B) Report both “timeout” and “0 results” as failures requiring coordinator intervention. (Correct: False)
C) Retry transient failures internally and report only persistent errors. (Correct: False)
D) Distinguish access failures (timeout) that require a retry decision from valid empty results (“0 results”) that represent successful queries. (Correct: True)

Correct Answer: D

Explanation (Gerekçe): Zaman aşımı (erişim hatası - access failure) ve ‘0 sonuç’ (geçerli boş sonuç - valid empty result) farklı aksiyonlar gerektiren, anlamsal olarak tamamen farklı sonuçlardır. Bunları birbirinden ayırt etmek, coordinator’ın patent veritabanını yeniden denemesini sağlarken, sektör raporlarındaki ‘0 sonuç’ durumunu geçerli ve bilgilendirici bir bulgu olarak kabul etmesine olanak tanır.

Q13: Multi-agent Research System (Scenario 13)

Situation: Production monitoring shows inconsistent synthesis quality. When aggregated results are ~75K tokens, the synthesis agent reliably cites information from the first 15K tokens (web-search headlines/snippets) and the last 10K tokens (document analysis conclusions), but often misses critical findings in the middle 50K tokens—even when they directly answer the research question. How should you restructure the aggregated input?

Question: How should you restructure the aggregated input?

Options:

A) Summarize all subagent outputs to under 20K tokens before aggregation to keep content within the model’s reliable processing range. (Correct: False)
B) Stream subagent results to the synthesis agent incrementally, processing web-search results first to completion, then adding document analysis results. (Correct: False)
C) Place a key-findings summary at the start of the aggregated input and organize detailed results with explicit section headings for easier navigation. (Correct: True)
D) Implement rotation that alternates which subagent’s results appear first across research tasks to ensure both sources get equal top positioning over time. (Correct: False)

Correct Answer: C

Explanation (Gerekçe): Girişin başına önemli bulguların özetini (key-findings summary) eklemek, öncelik etkisinden (primacy effects) yararlanarak kritik bilgilerin en güvenilir şekilde işlenen konuma yerleşmesini sağlar. Genel metin boyunca belirgin bölüm başlıkları (section headings) eklemek, modelin girdinin ortasındaki içeriğe odaklanmasına ve bu içerikte gezinmesine yardımcı olur; böylece ‘ortada kaybolma’ (lost in the middle) olgusunu doğrudan hafifletir.

Q14: Multi-agent Research System (Scenario 14)

Situation: In testing, the combined output of the web-search agent (85K tokens including page content) and the document analysis agent (70K tokens including chains of thought) totals 155K tokens, but the synthesis agent performs best with inputs under 50K tokens. Which solution is most effective?

Question: Which solution is most effective?

Options:

A) Modify upstream agents to return structured data (key facts, quotes, relevance scores) instead of verbose content and reasoning. (Correct: True)
B) Add an intermediate summarization agent that condenses findings before passing them to synthesis. (Correct: False)
C) Have the synthesis agent process findings in sequential batches, maintaining state between calls. (Correct: False)
D) Store findings in a vector database and give the synthesis agent search tools to query during its work. (Correct: False)

Correct Answer: A

Explanation (Gerekçe): Üst akıştaki (upstream) agent’ların yapılandırılmış veriler (önemli gerçekler, alıntılar, alaka puanları) döndürecek şekilde değiştirilmesi, temel bilgileri korurken token hacmini kaynakta azaltarak kök nedeni çözer. Bu yaklaşım, sentez aşamasını (synthesis step) iyileştirmeden token miktarını şişiren hacimli sayfa içeriklerinin ve akıl yürütme izlerinin (reasoning traces) aktarılmasını engeller.

Q15: Multi-agent Research System (Scenario 15)

Situation: In testing, you observe that the synthesis agent often needs to verify specific claims while merging results. Currently, when verification is needed, the synthesis agent returns control to the coordinator, which calls the web-search agent and then re-invokes synthesis with the results. This adds 2–3 extra loops per task and increases latency by 40%. Your assessment shows 85% of these verifications are simple fact checks (dates, names, stats) and 15% require deeper research. Which approach most effectively reduces overhead while preserving system reliability?

Question: Which approach is most effective?

Options:

A) Give the synthesis agent access to all web-search tools so it can handle any verification need directly without coordinator loops. (Correct: False)
B) Have the synthesis agent accumulate all verification needs and return them as a batch to the coordinator at the end, which then sends them all to the web-search agent at once. (Correct: False)
C) Have the web-search agent proactively cache extra context around each source during initial research in anticipation of synthesis needing verification. (Correct: False)
D) Give the synthesis agent a limited-scope verify_fact tool for simple checks, while routing complex verifications through the coordinator to the web-search agent. (Correct: True)

Correct Answer: D

Explanation (Gerekçe): Sınırlı kapsamlı bir olgu doğrulama (fact-verification) aracı (verify_fact), synthesis agent’ının basit kontrollerin %85’ini doğrudan gerçekleştirmesini sağlayarak döngülerin çoğunu ortadan kaldırır; aynı zamanda karmaşık doğrulamaların %15’i için coordinator delegasyon yolunu korur. Bu yöntem, en az yetki (least privilege) prensibini uygularken gecikmeyi (latency) de önemli ölçüde azaltır.

Q16 (Scenario: Claude Code for Continuous Integration)

Situation: Your CI pipeline runs the Claude Code CLI (in --print mode) using CLAUDE.md to provide project context for code review, and developers generally find the reviews substantive. However, they report that integrating findings into the workflow is difficult—Claude outputs narrative paragraphs that must be manually copied into PR comments. The team wants to automatically post each finding as a separate inline PR comment at the relevant place in code, which requires structured data with file path, line number, severity level, and suggested fix. Which approach is most effective?

Question: Which approach is most effective?

Options:

A: Add an “Output Format for Review” section to CLAUDE.md with examples of structured findings so Claude learns the expected format from project context.
B: Use the CLI flags --output-format json and --json-schema to enforce structured findings, then parse the output to post inline comments via the GitHub API.
C: Include explicit formatting instructions in the review prompt requiring each finding to follow a parseable template like [FILE:path] [LINE:n] [SEVERITY:level] ....
D: Keep narrative review format but add a summarization step that uses Claude to generate a structured JSON summary of findings.

Explanation (Türkçe): --output-format json ve --json-schema kullanımı, yapılandırılmış çıktıyı CLI düzeyinde zorunlu kılar; bu da GitHub API’si aracılığıyla satır içi (inline) PR yorumları olarak güvenilir bir şekilde ayrıştırılıp paylaşılabilecek, gerekli alanları (dosya yolu, satır numarası, önem derecesi, önerilen düzeltme) içeren düzgün biçimlendirilmiş (well-formed) bir JSON garanti eder. Bu yaklaşım, yapılandırılmış çıktılar için özel olarak tasarlanmış yerleşik CLI yeteneklerinden yararlanır.

Q17 (Scenario: Claude Code for Continuous Integration)

Situation: Your team uses Claude Code for generating code suggestions, but you notice a pattern: non-obvious issues—performance optimizations that break edge cases, cleanups that unexpectedly change behavior—are only caught when another team member reviews the PR. Claude’s reasoning during generation shows it considered these cases but concluded its approach was correct. Which approach directly addresses the root cause of this self-check limitation?

Question: Which approach directly addresses the root cause?

Options:

A: Run a second independent instance of Claude Code to review the changes without access to the generator’s reasoning.
B: Enable extended thinking mode for the generation stage to allow more thorough deliberation before producing suggestions.
C: Add explicit self-review instructions to the generation prompt asking Claude to critique its own suggestions before finalizing output.
D: Include full test files and documentation in prompt context so Claude better understands expected behavior during generation.

Explanation (Türkçe): Üretici ajanın (generator) akıl yürütme (reasoning) sürecine erişimi olmayan ikinci bir bağımsız Claude Code örneği çalıştırmak, doğrulama yanlılığını (confirmation bias) önleyerek kök nedeni doğrudan ele alır. Bu “taze gözle bakış” perspektifi, yazarın rasyonalize ettiği (kendi kendine haklı çıkardığı) sorunları başka bir gözden geçirenin yakaladığı insan akran denetimini (peer review) yansıtır.

Q18 (Scenario: Claude Code for Continuous Integration)

Situation: Your code review component is iterative: Claude analyzes the changed file, then may request related files (imports, base classes, tests) via tool calls to understand context before providing final feedback. Your application defines a tool that lets Claude request file contents; Claude calls the tool, gets results, and continues analysis. You’re evaluating batch processing to reduce API cost. What is the primary technical limitation when considering batch processing for this workflow?

Question: What is the primary technical limitation?

Options:

A: Batch processing does not include correlation IDs to map outputs back to input requests.
B: The asynchronous model cannot execute tools mid-request and return results for Claude to continue analysis.
C: The Batch API does not support tool definitions in request parameters.
D: The batch processing latency of up to 24 hours is too slow for pull request feedback, although the workflow would otherwise function.

Explanation (Türkçe): “Gönder ve unut” (fire-and-forget) tarzı asenkron bir Batch API modeli, bir istek sırasında gelen bir tool call’u kesintiye uğratacak, aracı (tool) çalıştıracak ve Claude’un analize devam etmesi için sonuçları döndürecek bir mekanizmaya sahip değildir. Bu durum, tek bir mantıksal etkileşim içinde birden fazla araç istek/yanıt turu (tool request/response rounds) gerektiren yinelemeli tool_use (araç çağırma) iş akışlarıyla temelden uyumsuzdur.

Q19 (Scenario: Claude Code for Continuous Integration)

Situation: Your CI/CD system runs three Claude-based analyses: (1) fast style checks on every PR that block merging until completion, (2) comprehensive weekly security audits of the entire codebase, and (3) nightly test-case generation for recently changed modules. The Message Batches API offers 50% savings but processing can take up to 24 hours. You want to optimize API cost while maintaining an acceptable developer experience. Which combination correctly matches each task to an API approach?

Question: Which combination is correct?

Options:

A: Use the Message Batches API for all three tasks to maximize 50% savings, configuring the pipeline to poll for batch completion.
B: Use synchronous calls for PR style checks; use the Message Batches API for weekly security audits and nightly test generation.
C: Use synchronous calls for all three tasks for consistent response times, relying on prompt caching to reduce costs across workloads.
D: Use synchronous calls for PR style checks and nightly test generation; use the Message Batches API only for weekly security audits.

Explanation (Türkçe): PR stil kontrolleri (style checks) geliştiricileri bloke eder ve senkron çağrılar (synchronous calls) aracılığıyla anında yanıt verilmesini gerektirir. Öte yandan, haftalık güvenlik denetimleri ve gecelik test oluşturma işlemleri, 24 saate kadar sürebilen batch penceresini tolere edebilecek esnek teslim tarihlerine sahip planlanmış görevlerdir; bu sayede her ikisinde de %50 tasarruf elde edilir.

Q20 (Scenario: Claude Code for Continuous Integration)

Situation: Your automated reviews find real issues, but developers report the feedback is not actionable. Findings include phrases like “complex ticket routing logic” or “potential null pointer” without specifying what exactly to change. When you add detailed instructions like “always include concrete fix suggestions,” the model still produces inconsistent output—sometimes detailed, sometimes vague. Which prompting technique most reliably produces consistently actionable feedback?

Question: Which prompting technique is most reliable?

Options:

A: Further refine instructions with more explicit requirements for each part of the feedback format (location, issue, severity, proposed fix).
B: Expand the context window to include more surrounding codebase so the model has enough information to propose concrete fixes.
C: Implement a two-pass approach where one prompt identifies issues and a second generates fixes, allowing specialization.
D: Add 3–4 few-shot examples showing the exact required format: identified issue, location in code, concrete fix suggestion.

Explanation (Türkçe): Few-shot örnekleri, tek başına talimatların değişken sonuçlar ürettiği durumlarda tutarlı bir çıktı formatı elde etmek için en etkili tekniktir. Tam olarak istenen yapıyı (sorun, kod içerisindeki konum, somut düzeltme önerisi) gösteren 3-4 örnek sunmak, modele takip edebileceği somut bir şablon sağlar ve bu, soyut talimatlara kıyasla çok daha güvenilirdir.

Q21 (Scenario: Claude Code for Continuous Integration)

Situation: Your CI pipeline includes two Claude-based code review modes: a pre-merge-commit hook that blocks PR merge until completion, and a “deep analysis” that runs overnight, polls for batch completion, and posts detailed suggestions to the PR. You want to reduce API cost using the Message Batches API, which offers 50% savings but requires polling and can take up to 24 hours. Which mode should use batch processing?

Question: Which mode should use batch processing?

Options:

A: Only the pre-merge-commit hook.
B: Only the deep analysis.
C: Both modes.
D: Neither mode.

Explanation (Türkçe): Derin analiz (deep analysis), halihazırda geceleri çalıştığı, gecikmeyi tolere edebildiği ve sonuçları yayınlamadan önce bir yoklama (polling) modeli kullandığı için batch işleme için ideal bir adaydır. Bu durum, Message Batches API’nin asenkron ve polling tabanlı mimarisiyle tam olarak eşleşirken %50 oranında tasarruf sağlar.

Q22 (Scenario: Claude Code for Continuous Integration)

Situation: Your automated review analyzes comments and docstrings. The current prompt instructs Claude to “check that comments are accurate and up to date.” Findings often flag acceptable patterns (TODO markers, simple descriptions) while missing comments describing behavior the code no longer implements. What change addresses the root cause of this inconsistent analysis?

Question: What change addresses the root cause?

Options:

A: Include git blame data so Claude can identify comments that predate recent code changes.
B: Add few-shot examples of misleading comments to help the model recognize similar patterns in the codebase.
C: Filter TODO, FIXME, and descriptive comment patterns before analysis to reduce noise.
D: Specify explicit criteria: flag comments only when the behavior they claim contradicts the code’s actual behavior.

Explanation (Türkçe): Belirgin kriterler koymak—yorumları yalnızca iddia edilen davranış kodun gerçek davranışı ile çeliştiğinde işaretlemek—belirsiz bir talimatı neyin sorun teşkil ettiğine dair kesin bir tanımla değiştirerek kök nedeni doğrudan ele alır. Bu, kabul edilebilir kalıplardaki yanlış pozitifleri (false positives) ve gerçekten yanıltıcı olan yorumların gözden kaçırılmasını azaltır.

Q23 (Scenario: Claude Code for Continuous Integration)

Situation: Your automated code review system shows inconsistent severity ratings—similar issues like null pointer risks are rated “critical” in some PRs but only “medium” in others. Developer surveys show growing distrust—many start dismissing findings without reading because “half are wrong.” High-false-positive categories erode trust in accurate categories. Which approach best restores developer trust while improving the system?

Question: Which approach best restores developer trust?

Options:

A: Temporarily disable high-false-positive categories (style, naming, documentation) and keep only high-precision categories while improving prompts.
B: Keep all categories enabled but display confidence scores with each finding so developers can decide what to investigate.
C: Keep all categories enabled and add few-shot examples to improve accuracy for each category over the next few weeks.
D: Apply a uniform strictness reduction across all categories to bring the overall false-positive rate down.

Explanation (Türkçe): Yüksek yanlış pozitif (false-positive) oranına sahip kategorileri geçici olarak devre dışı bırakmak, geliştiricilerin her şeyi göz ardı etmesine neden olan gürültülü bulguları ortadan kaldırarak güven kaybını anında durdurur; aynı zamanda güvenlik ve doğruluk gibi yüksek hassasiyetli kategorilerden elde edilen değeri korur. Bu aynı zamanda, sorunlu kategorilerin prompt’larını yeniden etkinleştirmeden önce iyileştirmek için alan yaratır.

Q24 (Scenario: Claude Code for Continuous Integration)

Situation: Your automated review generates test-case suggestions for each PR. Reviewing a PR that adds course completion tracking, Claude suggests 10 test cases, but developer feedback shows that 6 duplicate scenarios already covered by the existing test suite. What change most effectively reduces duplicate suggestions?

Question: What change is most effective?

Options:

A: Include the existing test file in context so Claude can determine what scenarios are already covered.
B: Reduce the requested number of suggestions from 10 to 5, assuming Claude prioritizes the most valuable cases first.
C: Add instructions directing Claude to focus exclusively on edge cases and error conditions rather than success paths.
D: Implement post-processing that filters suggestions whose descriptions match existing test names via keyword overlap.

Explanation (Türkçe): Mevcut test dosyasını bağlama (context) dahil etmek, tekrarlamanın kök nedenini çözer: Claude, halihazırda kapsanan senaryoları önermekten ancak hangi testlerin zaten var olduğunu bilirse kaçınabilir. Bu, Claude’a gerçekten yeni ve değerli testler önermesi için gereken bilgiyi sağlar.

Q25 (Scenario: Claude Code for Continuous Integration)

Situation: After an initial automated review identifies 12 findings, a developer pushes new commits to address issues. Re-running review produces 8 findings, but developers report that 5 duplicate previous comments on code that was already fixed in the new commits. What is the most effective way to eliminate this redundant feedback while maintaining thoroughness?

Question: What is the most effective way to eliminate redundant feedback?

Options:

A: Run review only when the PR is created and in the final pre-merge state, skipping intermediate commits.
B: Add a post-processing filter that removes findings that match previous ones by file paths and issue descriptions before posting comments.
C: Restrict review scope to files changed in the most recent push, excluding files from earlier commits.
D: Include previous review findings in context and instruct Claude to report only new or still-unresolved issues.

Explanation (Türkçe): Önceki inceleme bulgularını bağlama (context) dahil etmek, Claude’un yeni sorunları, son commit’lerde zaten çözülmüş olanlardan ayırt etmesini sağlar. Bu durum, düzeltilmiş kod üzerinde gereksiz geri bildirimlerden kaçınmak için Claude’un akıl yürütme (reasoning) yeteneğini kullanırken inceleme kapsamlılığını da korur.

Q26 (Scenario: Claude Code for Continuous Integration)

Situation: Your pipeline script runs claude "Analyze this pull request for security issues", but the job hangs indefinitely. Logs show Claude Code is waiting for interactive input. What is the correct approach to run Claude Code in an automated pipeline?

Question: What is the correct approach?

Options:

A: Add a --batch flag: claude --batch "Analyze this pull request for security issues".
B: Add the -p flag: claude -p "Analyze this pull request for security issues".
C: Redirect stdin from /dev/null: claude "Analyze this pull request for security issues" < /dev/null.
D: Set the environment variable CLAUDE_HEADLESS=true before running the command.

Explanation (Türkçe): -p (veya --print) bayrağı, Claude Code’u etkileşimli olmayan (non-interactive) bir şekilde çalıştırmak için belgelenmiş yöntemdir. Prompt’u işler, sonucu stdout’a yazdırır ve kullanıcı girişini beklemeden çıkış yapar; bu da CI/CD boru hatları (pipelines) için idealdir.

Q27 (Scenario: Claude Code for Continuous Integration)

Situation: A pull request changes 14 files in an inventory tracking module. A single-pass review that analyzes all files together produces inconsistent results: detailed feedback on some files but shallow comments on others, missed obvious bugs, and contradictory feedback (a pattern is flagged in one file but identical code is approved in another file in the same PR). How should you restructure the review?

Question: How should you restructure the review?

Options:

A: Run three independent full-PR review passes and flag only issues that appear in at least two of the three runs.
B: Split into focused passes: review each file individually for local issues, then run a separate integration-oriented pass to examine cross-file data flows.
C: Require developers to split large PRs into smaller submissions of 3–4 files before running automated review.
D: Switch to a larger model with a bigger context window so it can pay sufficient attention to all 14 files in one pass.

Explanation (Türkçe): Dosya başına odaklanmış geçişler (passes), tutarlı derinlik ve güvenilir yerel sorun tespiti sağlayarak kök nedeni—dikkat dağılmasını (attention dilution)—ele alır. Ayrı bir entegrasyon odaklı geçiş ise bağımlılık ve veri akışı (data-flow) etkileşimleri gibi dosyalar arası sorunları kapsar.

Q28 (Scenario: Claude Code for Continuous Integration)

Situation: Your automated code review averages 15 findings per pull request, and developers report a 40% false-positive rate. The bottleneck is investigation time: developers must click into each finding to read Claude’s rationale before deciding whether to fix or dismiss it. Your CLAUDE.md already contains comprehensive rules for acceptable patterns, and stakeholders rejected any approach that filters findings before developers see them. What change best addresses investigation time?

Question: What change best addresses investigation time?

Options:

A: Require Claude to include its rationale and confidence estimate directly in each finding.
B: Add a post-processor that analyzes finding patterns and automatically suppresses those that match historical false-positive signatures.
C: Categorize findings as “blocking issues” vs “suggestions,” with different review requirements by level.
D: Configure Claude to show only high-confidence findings, filtering uncertain flags before developers see them.

Explanation (Türkçe): Gerekçeyi ve güven tahminini (confidence estimate) doğrudan her bulguya dahil etmek, geliştiricilerin her bulguyu açmak zorunda kalmadan hızlı bir şekilde önceliklendirme (triage) yapmasına olanak tanıyarak inceleme süresini azaltır. Tüm bulgular görünür kalırken geliştiricinin karar verme sürecini hızlandırdığı için “filtreleme yapmama” kısıtlamasını karşılar.

Q29 (Scenario: Claude Code for Continuous Integration)

Situation: Analysis of your automated code review shows large differences in false-positive rates by finding category: security/correctness findings have 8% false positives, performance findings 18%, style/naming findings 52%, and documentation findings 48%. Developer surveys show growing distrust—many start dismissing findings without reading because “half are wrong.” High-false-positive categories erode trust in accurate categories. Which approach best restores developer trust while improving the system?

Question: Which approach best restores developer trust?

Options:

A: Temporarily disable high-false-positive categories (style, naming, documentation) and keep only high-precision categories while improving prompts.
B: Keep all categories enabled but display confidence scores with each finding so developers can decide what to investigate.
C: Keep all categories enabled and add few-shot examples to improve accuracy for each category over the next few weeks.
D: Apply a uniform strictness reduction across all categories to bring the overall false-positive rate down.

Q30 (Scenario: Claude Code for Continuous Integration)

Situation: Your team wants to reduce API costs for automated analysis. Currently, synchronous Claude calls support two workflows: (1) a blocking pre-merge check that must complete before developers can merge, and (2) a technical debt report generated overnight for review the next morning. Your manager proposes moving both to the Message Batches API to save 50%. How should you evaluate this proposal?

Question: How should you evaluate this proposal?

Options:

A: Move both to batch processing with fallback to synchronous calls if batches take too long.
B: Move both workflows to batch processing with status polling to verify completion.
C: Use batch processing only for technical debt reports; keep synchronous calls for pre-merge checks.
D: Keep synchronous calls for both workflows to avoid issues with batch result ordering.

Explanation (Türkçe): Message Batches API işlemleri, herhangi bir gecikme SLA’i olmaksızın 24 saate kadar sürebilir; bu durum gecelik teknik borç (technical debt) raporları için kabul edilebilir olsa da geliştiricilerin beklediği engelleyici pre-merge kontrolleri için kabul edilemez. Bu durum, gecikme (latency) gereksinimlerine bağlı olarak her iş akışını doğru API ile eşleştirir.

Q31 (Scenario: Code Generation with Claude Code)

Situation: You asked Claude Code to implement a function that transforms API responses into an internal normalized format. After two iterations, the output structure still doesn’t match expectations—some fields are nested differently and timestamps are formatted incorrectly. You described requirements in prose, but Claude interprets them differently each time.

Question: Which approach is most effective for the next iteration?

Options:

A. Write a JSON schema describing the expected output structure and validate Claude’s output against it after each iteration.
B. (Correct) Provide 2–3 concrete input-output examples showing the expected transformation for representative API responses.
C. Rewrite requirements with more technical precision, specifying exact field mappings, nesting rules, and timestamp format strings.
D. Ask Claude to explain its current understanding of the requirements to identify where interpretations diverge.

Explanation (Türkçe): Somut girdi-çıktı örnekleri, Claude’a tam olarak beklenen dönüşüm sonuçlarını göstererek düz metin (prose) açıklamalarında bulunan belirsizlikleri ortadan kaldırır. Bu yaklaşım, alanların iç içe geçmesi (nesting) ve zaman damgası (timestamp) biçimlendirmesi için net/belirsiz olmayan şablonlar (patterns) sağlayarak, metinsel gereksinimlerin yanlış yorumlanması şeklindeki kök nedeni (root cause) doğrudan ele alır.

Q32 (Scenario: Code Generation with Claude Code)

Situation: You need to add Slack as a new notification channel. The existing codebase has clear, established patterns for email, SMS, and push channels. However, Slack’s API offers fundamentally different integration approaches—incoming webhooks (simple, one-way), bot tokens (support delivery confirmation and programmatic control), or Slack Apps (two-way events, requires workspace approval). Your task says “add Slack support” without specifying integration method or requiring advanced features like delivery tracking.

Question: How should you approach this task?

Options:

A. Start in direct execution mode using incoming webhooks to match the existing one-way notification pattern.
B. (Correct) Switch to planning mode to explore integration options and architectural implications, then present a recommendation before implementation.
C. Start in direct execution mode by scaffolding a Slack channel class using existing patterns, deferring the integration method decision.
D. Start in direct execution mode using a bot-token approach to ensure delivery confirmation is possible.

Explanation (Türkçe): Slack entegrasyonu, mimari açıdan önemli ölçüde farklı etkileri olan birden fazla geçerli yaklaşıma sahiptir ve gereksinimler belirsizdir. Planlama modu (planning mode), webhooks, bot token’ları ve Slack App’leri arasındaki ödünleşimleri (trade-offs) değerlendirmenize ve uygulamaya geçmeden önce bir yaklaşım üzerinde mutabık kalmanıza olanak tanır.

Q33 (Scenario: Code Generation with Claude Code)

Situation: Your CLAUDE.md file has grown to 400+ lines containing coding standards, testing conventions, a detailed PR review checklist, deployment instructions, and database migration procedures. You want Claude to always follow coding standards and testing conventions, but apply PR review, deploy, and migration guidance only when doing those tasks.

Question: Which restructuring approach is most effective?

Options:

A. Move all guidance into separate Skills files organized by workflow type, leaving only a brief project description in CLAUDE.md.
B. Keep everything in CLAUDE.md but use @import syntax to organize into separately maintained files by category.
C. Split CLAUDE.md into files under .claude/rules/ with path-bound glob patterns so each rule loads only for the relevant file types.
D. (Correct) Keep universal standards in CLAUDE.md and create Skills for workflow-specific guidance (PR review, deploy, migrations) with trigger keywords.

Explanation (Türkçe): CLAUDE.md içeriği her oturumda yüklenerek kodlama standartlarının ve test kurallarının (testing conventions) her zaman uygulanmasını sağlarken, Skills ise Claude tetikleyici anahtar kelimeleri (trigger keywords) algıladığında talep üzerine çağrılır. Bu, PR incelemesi, dağıtım (deploy) ve veritabanı geçişleri (migrations) gibi iş akışına özel yönlendirmeler için idealdir.

Q34 (Scenario: Code Generation with Claude Code)

Situation: You’re tasked with restructuring your team’s monolithic application into microservices. This impacts changes across dozens of files and requires decisions about service boundaries and module dependencies.

Question: Which approach should you choose?

Options:

A. (Correct) Switch to planning mode to explore the codebase, understand dependencies, and design the implementation approach before making changes.
B. Start in direct execution mode and switch to planning only after encountering unexpected complexity during implementation.
C. Start in direct execution mode and make incremental changes, letting implementation reveal natural service boundaries.
D. Use direct execution with detailed upfront instructions that specify each service structure.

Explanation (Türkçe): Planlama modu (planning mode), monolitik bir yapıyı bölmek gibi karmaşık mimari yeniden yapılandırmalar için doğru stratejidir: birçok dosyada potansiyel olarak maliyetli değişiklikler yapmadan önce güvenli bir keşif yapılmasına ve sınırlar hakkında bilinçli kararlar alınmasına olanak tanır.

Q35 (Scenario: Code Generation with Claude Code)

Situation: Your team created a /analyze-codebase skill that performs deep code analysis—dependency scanning, test coverage counts, and code quality metrics. After running the command, team members report Claude becomes less responsive in the session and loses the context of the original task.

Question: How do you most effectively fix this while keeping full analysis capabilities?

Options:

A. (Correct) Add context: fork in the skill frontmatter to run the analysis in an isolated subagent context.
B. Add model: haiku in frontmatter to use a faster, cheaper model for analysis.
C. Split the skill into three smaller skills, each producing less output.
D. Add instructions to the skill to compress all results into a short summary before displaying them.

Explanation (Türkçe): context: fork ayarı, analizi izole edilmiş bir subagent bağlamında (context) çalıştırır; böylece büyük çıktılar ana oturumun bağlam penceresini (context window) kirletmez ve Claude orijinal görevin takibini kaybetmez. Ana oturumu duyarlı (responsive) tutarken tam analiz yeteneğini korur.

Q36 (Scenario: Code Generation with Claude Code)

Situation: Your team uses a /commit skill in .claude/skills/commit/SKILL.md. A developer wants to customize it for their personal workflow (different commit message format, extra checks) without affecting teammates.

Question: What do you recommend?

Options:

A. Create a personal version under ~/.claude/skills/ with a different name, e.g., /my-commit.
B. Add conditional logic based on username in the project skill frontmatter.
C. (Correct) Create a personal version at ~/.claude/skills/commit/SKILL.md with the same name.
D. Set override: true in the personal skill frontmatter to prioritize it over the project version.

Explanation (Türkçe): Kişisel yetenekler (personal skills), aynı adı taşıyan proje yeteneklerine (project skills) göre önceliğe sahiptir. ~/.claude/skills/commit/SKILL.md konumundaki kişisel bir skill, ekibin proje skill’ini geçersiz kılarak (override), geliştiricinin kendi kullanımı için tanıdık /commit komut adını korurken iş akışını özelleştirmesine olanak tanır. Bu yaklaşım, A seçeneğine göre daha iyidir çünkü orijinal komut adını korur ve ekip arkadaşlarını etkilemeden geliştiricinin iş akışını iyileştirir.

Q37 (Scenario: Code Generation with Claude Code)

Situation: Your team has used Claude Code for months. Recently, three developers report Claude follows the guidance “always include comprehensive error handling,” but a fourth developer who just joined says Claude does not follow it. All four work in the same repo and have up-to-date code.

Question: What is the most likely cause and fix?

Options:

A. (Correct) The guidance lives in the original developers’ user-level ~/.claude/CLAUDE.md files, not in the project .claude/CLAUDE.md. Move the instruction to the project-level file so all team members receive it.
B. The new developer’s ~/.claude/CLAUDE.md contains conflicting instructions overriding project settings; they should delete the conflicting section.
C. Claude Code learns per-user preferences over time; the new developer must repeat the requirement until Claude “remembers” it.
D. Claude Code caches CLAUDE.md after first read; original developers use cached versions. Everyone should clear the Claude Code cache.

Explanation (Türkçe): Yönlendirme yalnızca orijinal geliştiricilerin kullanıcı düzeyindeki konfigürasyonlarına eklenmiş ve proje düzeyindeki .claude/CLAUDE.md dosyasına eklenmemişse, yeni ekip üyeleri bu yönlendirmeyi alamaz. Bunu proje düzeyindeki konfigürasyona taşımak, mevcut ve gelecekteki tüm ekip üyelerinin bu yönlendirmeyi otomatik olarak almasını sağlar.

Q38 (Scenario: Code Generation with Claude Code)

Situation: You find that including 2–3 full endpoint implementation examples as context significantly improves consistency when generating new API endpoints. However, this context is useful only when creating new endpoints—not when debugging, reviewing code, or other work in the API directory.

Question: Which configuration approach is most effective?

Options:

A. Add endpoint examples and pattern documentation to the project CLAUDE.md so they are always available.
B. Manually reference endpoint examples in every generation request by copying code into the prompt.
C. Configure path-specific rules in .claude/rules/api/ that include endpoint examples and activate when working in the API directory.
D. (Correct) Create a skill that references the endpoint examples and contains pattern-following instructions, invoked on demand via a slash command.

Explanation (Türkçe): Talep üzerine çağrılan (invoked on demand) bir skill, örnek bağlamını (context) yalnızca yeni uç noktalar (endpoints) üretilirken yükler; hata ayıklama (debugging) veya inceleme gibi ilgisiz görevler sırasında yüklemez. Bu, gerektiğinde yüksek kaliteli kod üretimini korurken ana bağlamı (context) temiz tutar.

Q39 (Scenario: Code Generation with Claude Code)

Situation: Your team created a /migration skill that generates database migration files. It takes the migration name via $ARGUMENTS. In production you observe three issues: (1) developers often run the skill without arguments, causing poorly named files, (2) the skill sometimes uses database schema details from unrelated prior conversations, and (3) a developer accidentally ran destructive test cleanup when the skill had broad tool access.

Question: Which configuration approach fixes all three problems?

Options:

A. Use positional parameters $1 and $2 instead of $ARGUMENTS to enforce specific inputs, include explicit schema file references via @ syntax for context control, and add a frontmatter description warning about destructive operations.
B. (Correct) Add argument-hint in frontmatter to request required parameters, use context: fork to isolate execution, and restrict allowed-tools to file-write operations.
C. Split into /migration-create and /migration-apply skills, add validation instructions to request migration name if missing, and use different allowed-tools scopes for each.
D. Add validation instructions in the skill SKILL.md to ensure $ARGUMENTS is a valid name, add prompts to ignore prior conversation context, and list prohibited operations to avoid.

Explanation (Türkçe): Bu çözüm, her bir sorunu çözmek için üç ayrı konfigürasyon özelliğini kullanır: argument-hint parametre girişini iyileştirir ve eksik argümanları azaltır, context: fork önceki konuşmalardan bağlam sızıntısını (context leakage) önler ve allowed-tools yeteneği (skill) güvenli dosya yazma işlemleriyle sınırlandırarak yıkıcı eylemleri engeller.

Q40 (Scenario: Code Generation with Claude Code)

Situation: Your codebase contains areas with different coding conventions: React components use functional style with hooks, API handlers use async/await with specific error handling, and database models follow the repository pattern. Test files are distributed across the codebase next to the code under test (e.g., Button.test.tsx next to Button.tsx), and you want all tests to follow the same conventions regardless of location.

Question: What is the most supported way to ensure Claude automatically applies the correct conventions when generating code?

Options:

A. Put all conventions in the root CLAUDE.md under headings for each area and rely on Claude to infer which section applies.
B. Create skills in .claude/skills/ for each code type, embedding conventions in each SKILL.md.
C. Place a separate CLAUDE.md file in each subdirectory containing conventions for that area.
D. (Correct) Create rule files under .claude/rules/ with YAML frontmatter specifying glob patterns to conditionally apply conventions based on file paths.

Explanation (Türkçe): YAML frontmatter ve glob şablonları (örneğin **/*.test.tsx, src/api/**/*.ts) içeren .claude/rules/ dosyaları, dizin yapısından bağımsız olarak dosya yollarına göre deterministik ve yol tabanlı kural (convention) uygulanmasını sağlar. Bu yaklaşım, dağıtılmış test dosyaları gibi ortak kesişen şablonlar (cross-cutting patterns) için en çok desteklenen yöntemdir.

Q41 (Scenario: Code Generation with Claude Code)

Situation: You want to create a custom slash command /review that runs your team’s standard code review checklist. It should be available to every developer when they clone or update the repository.

Question: Where should you create the command file?

Options:

A. In ~/.claude/commands/ in each developer’s home directory.
B. (Correct) In the project repository under .claude/commands/.
C. In .claude/config.json as an array of commands.
D. In the root project CLAUDE.md.

Explanation (Türkçe): Özel eğik çizgi komutlarını (custom slash commands) proje deposundaki .claude/commands/ dizini altına yerleştirmek, bunların sürüm kontrollü olmasını ve depoyu klonlayan veya güncelleyen her geliştirici için otomatik olarak kullanılabilir olmasını sağlar. Bu konum, Claude Code içindeki proje düzeyindeki özel komutlar için tasarlanmış yerdir.

Q42 (Scenario: Code Generation with Claude Code)

Situation: Your team’s CLAUDE.md grew beyond 500 lines mixing TypeScript conventions, testing guidance, API patterns, and deployment procedures. Developers find it hard to locate and update the right sections.

Question: What approach does Claude Code support to organize project-level instructions into focused topical modules?

Options:

A. Define a .claude/config.yaml mapping file patterns to specific sections inside CLAUDE.md.
B. (Correct) Create separate Markdown files in .claude/rules/, each covering one topic (e.g., testing.md, api-conventions.md).
C. Split instructions into README.md files in relevant subdirectories that Claude automatically loads as instructions.
D. Create multiple files named CLAUDE.md at different levels of the directory tree, each overriding parent instructions.

Explanation (Türkçe): Claude Code, konu odaklı rehberlik için ayrı Markdown dosyaları (örneğin testing.md, api-conventions.md) oluşturabileceğiniz bir .claude/rules/ dizinini destekler; bu da ekiplerin büyük talimat setlerini odaklanmış ve sürdürülebilir modüller halinde organize etmesine olanak tanır.

Q43 (Scenario: Code Generation with Claude Code)

Situation: You create a custom skill /explore-alternatives that your team uses to brainstorm and evaluate implementation approaches before choosing one. Developers report that after running the skill, subsequent Claude responses are influenced by the alternatives discussion—sometimes referencing rejected approaches or retaining exploration context that interferes with actual implementation.

Question: How should you most effectively configure this skill?

Options:

A. Use the ! prefix in the skill to run exploration logic as a bash subprocess.
B. (Correct) Add context: fork in the skill frontmatter.
C. Split into two skills—/explore-start and /explore-end—to mark boundaries when exploration context should be discarded.
D. Create the skill in ~/.claude/skills/ instead of .claude/skills/.

Explanation (Türkçe): context: fork ayarı, yeteneği (skill) izole edilmiş bir subagent bağlamında (context) çalıştırır, böylece keşif tartışmaları ana konuşma geçmişini kirletmez. Bu sayede reddedilen yaklaşımların ve beyin fırtınası bağlamının daha sonraki uygulama (implementation) çalışmalarını etkilemesi önlenir.

Q44 (Scenario: Code Generation with Claude Code)

Situation: Your team wants to add a GitHub MCP server for searching PRs and checking CI status via Claude Code. Each of six developers has their own personal GitHub access token. You want consistent tooling across the team without committing credentials to version control.

Question: Which configuration approach is most effective?

Options:

A. Have each developer add the server in user scope via claude mcp add --scope user.
B. Create an MCP server wrapper that reads tokens from a .env file and proxies GitHub API calls, then add the wrapper to the project .mcp.json.
C. (Correct) Add the server to the project .mcp.json using environment variable substitution (${GITHUB_TOKEN}) for auth and document the required environment variable in the project README.
D. Configure the server in project scope with a placeholder token, then tell developers to override it in their local config.

Explanation (Türkçe): Ortam değişkeni ikamesi (environment variable substitution) kullanan bir proje .mcp.json dosyası kullanımı idiomatiktir: MCP yapılandırması için sürüm kontrollü tek bir doğruluk kaynağı (source of truth) sunarken, her geliştiricinin kimlik bilgilerini ortam değişkenleri (environment variables) aracılığıyla sağlamasına izin verir. Değişkenin belgelenmesi, sırları (secrets) versiyon kontrol sistemine taahhüt etmeden (committing) yeni geliştiricilerin sisteme dahil olmasını (onboarding) kolaylaştırır.

Q45 (Scenario: Code Generation with Claude Code)

Situation: You’re adding error-handling wrappers around external API calls across a 120-file codebase. The work has three phases: (1) discover all call sites and patterns, (2) collaboratively design the error-handling approach, and (3) implement wrappers consistently. In Phase 1, Claude generates large output listing hundreds of call sites with context, quickly filling the context window before discovery finishes.

Question: Which approach is most effective to complete the task while maintaining implementation consistency?

Options:

A. (Correct) Use an Explore subagent for Phase 1 to isolate verbose discovery output and return a summary, then continue Phases 2–3 in the main conversation.
B. Do all phases in the main conversation, periodically using /compact to reduce context usage while moving through files.
C. Switch to headless mode with --continue, passing explicit context summaries between batch calls to maintain continuity.
D. Define the error-handling pattern in CLAUDE.md, then process files in batches across multiple sessions relying on the shared memory file for consistency.

Explanation (Türkçe): Bir Explore subagent’ı, ayrıntılı keşif (discovery) çıktısını ayrı bir bağlamda (context) izole eder ve ana konuşmaya yalnızca kısa bir özet döndürür. Bu sayede, tutulan bağlamın en değerli olduğu ortak tasarım (collaborative design) ve tutarlı uygulama (implementation) aşamaları için ana bağlam penceresi (context window) korunmuş olur.

Q46 (Scenario: Customer Support Agent)

Situation

While testing, you notice the agent often calls get_customer when users ask about order status, even though lookup_order would be more appropriate. What should you check first to address this problem?

Question

What should you check first?

Options

A) Implement a preprocessing classifier to detect order-related requests and route them directly to lookup_order.
B) Reduce the number of tools available to the agent to simplify choice.
C) Add few-shot examples to the system prompt covering all possible order request patterns to improve tool selection.
D) Check the tool descriptions to ensure they clearly differentiate each tool’s purpose. (Correct)

Explanation (Türkçe)

Tool açıklamaları, modelin hangi tool’u çağıracağına karar vermek için kullandığı birincil girdidir. Bir agent sürekli olarak yanlış tool’u seçtiğinde, ilk teşhis adımı tool açıklamalarının her bir tool’un amacını ve kullanım sınırlarını net bir şekilde ayırt ettiğini doğrulamaktır.

Q47 (Scenario: Customer Support Agent)

Situation

Your agent handles single-issue requests with 94% accuracy (e.g., “I need a refund for order #1234”). But when customers include multiple issues in one message (e.g., “I need a refund for order #1234 and also want to update the shipping address for order #5678”), tool selection accuracy drops to 58%. The agent usually solves only one issue or mixes parameters across requests. What approach most effectively improves reliability for multi-issue requests?

Question

What approach is most effective?

Options

A) Implement a preprocessing layer that uses a separate model call to decompose multi-issue messages into separate requests, handle each independently, and merge results.
B) Combine related tools into fewer universal tools.
C) Add few-shot examples to the prompt demonstrating correct reasoning and tool sequencing for multi-issue requests. (Correct)
D) Implement response validation that detects incomplete answers and automatically reprompts the agent to resolve missed issues.

Explanation (Türkçe)

Çoklu sorun (multi-issue) talepleri için doğru muhakeme (reasoning) ve tool sıralamasını (sequencing) gösteren few-shot örnekleri en etkili yöntemdir; çünkü agent tekil sorunlarda zaten iyi performans göstermektedir—ihtiyacı olan şey, çoklu sorunları ayrıştırma, yönlendirme ve parametreleri ayrı tutma örüntüsüne yönelik rehberliktir.

Q48 (Scenario: Customer Support Agent)

Situation

Production logs show that for simple requests like “refund for order #1234,” your agent resolves the issue in 3–4 tool calls with 91% success. But for complex requests like “I was billed twice, my discount didn’t apply, and I want to cancel,” the agent averages 12+ tool calls with only 54% success—often investigating issues sequentially and fetching redundant customer data for each. What change most effectively improves handling of complex requests?

Question

What change is most effective?

Options

A) Add explicit verification checkpoints between stages, requiring the agent to record progress after resolving each issue before moving to the next.
B) Reduce the number of tools by combining get_customer, lookup_order, and billing-related tools into a single investigate_issue tool.
C) Decompose the request into separate issues, then investigate each in parallel using shared customer context before synthesizing a final resolution. (Correct)
D) Add few-shot examples to the system prompt demonstrating ideal tool-call sequences for various multi-faceted billing scenarios.

Explanation (Türkçe)

Talebi ayrı sorunlara ayrıştırmak (decompose) ve ortak bir müşteri bağlamı (customer context) kullanarak bunları paralel olarak incelemek her iki temel sorunu da çözer: sorunlar arasında paylaşılan bağlamı yeniden kullanarak mükerrer veri alımını ortadan kaldırır ve tek bir nihai çözüm sentezlemeden önce araştırmayı paralel hale getirerek toplam tool-call döngülerini azaltır.

Q49 (Scenario: Customer Support Agent)

Situation

Your agent achieves 55% first-contact resolution, well below the 80% target. Logs show it escalates simple cases (standard replacements for damaged goods with photo proof) while trying to handle complex situations requiring policy exceptions autonomously. What is the most effective way to improve escalation calibration?

Question

What is the most effective way to improve escalation calibration?

Options

A) Require the agent to self-rate confidence on a 1–10 scale before each response and automatically route to humans when confidence drops below a threshold.
B) Deploy a separate classifier model trained on historical tickets to predict which requests need escalation before the main agent starts processing.
C) Add explicit escalation criteria to the system prompt with few-shot examples showing when to escalate versus resolve autonomously. (Correct)
D) Implement sentiment analysis to determine customer frustration level and automatically escalate past a negative sentiment threshold.

Explanation (Türkçe)

Few-shot örneklerle birlikte açık escalation kriterleri, basit ve karmaşık durumlar arasındaki belirsiz karar sınırları olan kök nedeni doğrudan ele alır. Bu, ek altyapı gerektirmeden agent’a ne zaman escalate edeceğini ve ne zaman otonom olarak çözeceğini öğreten en orantılı ve etkili ilk müdahaledir.

Q50 (Scenario: Customer Support Agent)

Situation

After calling get_customer and lookup_order, the agent has all available system data but still faces uncertainty. Which situation is the most justified trigger for calling escalate_to_human?

Question

Which situation is most justified for escalation?

Options

A) A customer wants to cancel an order shipped yesterday and arriving tomorrow. The agent should escalate because the customer might change their mind after receiving the package.
B) A customer claims they didn’t receive an order, but tracking shows it was delivered and signed for at their address three days ago. The agent should escalate because presenting contradictory evidence could harm the customer relationship.
C) A customer requests competitor price matching. Your policies allow price adjustments for price drops on your own site within 14 days, but say nothing about competitor prices. The agent should escalate for policy interpretation. (Correct)
D) A customer message contains both a billing question and a product return. The agent should escalate so a human can coordinate both issues in one interaction.

Explanation (Türkçe)

Bu, gerçek bir politika boşluğudur (policy gap): şirket kuralları kendi sitenizdeki fiyat düşüşlerini kapsamakta ancak rakip fiyat eşleştirmesini (competitor price matching) ele almamaktadır. Agent politika uydurmamalı ve mevcut kuralların nasıl yorumlanacağı veya genişletileceği konusunda insani değerlendirme (human judgment) için durumu escalate etmelidir.

Q51 (Scenario: Customer Support Agent)

Situation

Production logs show that in 12% of cases your agent skips get_customer and calls lookup_order directly using only the customer-provided name, sometimes leading to misidentified accounts and incorrect refunds. What change most effectively fixes this reliability problem?

Question

What change is most effective?

Options

A) Add few-shot examples showing that the agent always calls get_customer first, even when customers voluntarily provide order details.
B) Implement a routing classifier that analyzes each request and enables only a subset of tools appropriate for that request type.
C) Add a programmatic precondition that blocks lookup_order and process_refund until get_customer returns a verified customer identifier. (Correct)
D) Strengthen the system prompt stating that customer verification via get_customer is mandatory before any order operations.

Explanation (Türkçe)

Programatik bir önkoşul (precondition), gerekli sıralamanın (sequencing) takip edildiğine dair deterministik bir garanti sağlar. LLM davranışından bağımsız olarak doğrulamanın atlanması olasılığını ortadan kaldırdığı için en etkili yaklaşımdır.

Q52 (Scenario: Customer Support Agent)

Situation

Production metrics show that when resolving complex billing disputes or multi-order returns, customer satisfaction scores are 15% lower than for simple cases—even when the resolution is technically correct. Root-cause analysis shows the agent provides accurate solutions but inconsistently explains rationale: sometimes omitting relevant policy details, sometimes missing timeline info or next steps. The specific context gaps vary case by case. You want to improve solution quality without adding human oversight. What approach is most effective?

Question

What approach is most effective?

Options

A) Add a self-critique stage where the agent evaluates a draft response for completeness—ensuring it resolves the customer’s issue, includes relevant context, and anticipates follow-up questions. (Correct)
B) Add a confirmation stage where the agent asks “Does this fully resolve your issue?” before closing, allowing customers to request additional information if needed.
C) Upgrade the model from Haiku to Sonnet for complex cases, routing based on a defined complexity metric.
D) Implement few-shot examples in the system prompt showing complete explanations for five common complex case types, demonstrating how to include policy context, timelines, and next steps.

Explanation (Türkçe)

Bir self-critique aşaması (evaluator-optimizer pattern), agent’ı yanıtı sunmadan önce kendi taslağını politika bağlamı (policy context), zaman çizelgeleri ve sonraki adımlar gibi somut kriterlere göre değerlendirmeye zorlayarak tutarsız açıklama bütünlüğünü doğrudan ele alır. Bu, insan gözetimi (human oversight) olmadan vakaya özel boşlukları yakalar.

Q53 (Scenario: Customer Support Agent)

Situation

Production metrics show your agent averages 4+ API loops per resolution. Analysis reveals Claude often requests get_customer and lookup_order in separate sequential turns even when both are needed initially. What is the most effective way to reduce the number of loops?

Question

What is the most effective way to reduce loops?

Options

A) Implement speculative execution that automatically calls likely-needed tools in parallel with any requested tool and returns all results regardless of what was requested.
B) Increase max_tokens to give Claude more room to plan and naturally combine tool requests.
C) Create composite tools like get_customer_with_orders that bundle common lookup combinations into single calls.
D) Instruct Claude in the prompt to bundle tool requests into one turn and return all results together before the next API call. (Correct)

Explanation (Türkçe)

Claude’a ilgili tool isteklerini tek bir turda paketlemesini (bundle) söylemek, onun aynı anda birden fazla tool talep etme konusundaki yerel yeteneğinden yararlanır. Minimum mimari değişiklik ile ardışık çağrı (sequential-call) modelini doğrudan düzeltir.

Q54 (Scenario: Customer Support Agent)

Situation

Production logs show a pattern: customers reference specific amounts (e.g., “the 15% discount I mentioned”), but the agent responds with incorrect values. Investigation shows these details were mentioned 20+ turns ago and condensed into vague summaries like “promotional pricing was discussed.” What fix is most effective?

Question

What fix is most effective?

Options

A) Increase the summarization threshold from 70% to 85% so conversations have more room before summarization triggers.
B) Store full conversation history in external storage and implement retrieval when the agent detects references like “as I mentioned.”
C) Extract transactional facts (amounts, dates, order numbers) into a persistent “case facts” block included in every prompt outside the summarized history. (Correct)
D) Revise the summarization prompt to explicitly preserve all numbers, percentages, dates, and customer-stated expectations verbatim.

Explanation (Türkçe)

Özetleme (summarization) doğası gereği kesin detayları kaybettirir. İşlemsel gerçekleri (transactional facts - tutarlar, tarihler, sipariş numaraları) özetlenen geçmişin dışındaki yapılandırılmış (structured) bir “case facts” bloğuna çıkarmak, kaç turun özetlendiğine bakılmaksızın kritik bilgilerin her prompt’ta güvenilir bir şekilde kullanılabilir olmasını sağlar.

Q55 (Scenario: Customer Support Agent)

Situation

Your get_customer tool returns all matches when searching by name. Currently, when there are multiple results, Claude picks the customer with the most recent order, but production data shows this selects the wrong account 15% of the time for ambiguous matches. How should you address this?

Question

How should you address this?

Options

A) Implement a confidence scoring system that acts autonomously above 85% confidence and requests clarification below the threshold.
B) Instruct Claude to request an additional identifier (email, phone, or order number) when get_customer returns multiple matches before taking any customer-specific action. (Correct)
C) Modify get_customer to return only a single most-likely match based on a ranking algorithm, eliminating ambiguity.
D) Add few-shot examples to the prompt demonstrating correct reasoning and tool sequencing for ambiguous matches.

Explanation (Türkçe)

Kullanıcıdan ek bir kimlik bilgisi (identifier) istemek, belirsizliği (ambiguity) çözmenin en güvenilir yoludur çünkü kullanıcı kendi kimliği hakkında kesin bilgiye sahiptir. Yanlış hesabın seçilmesinden kaynaklanan %15’lik hata oranını ortadan kaldırmak için bir ek konuşma turu (conversational turn) küçük bir bedeldir.

Q56 (Scenario: Customer Support Agent)

Situation

Production logs show a consistent pattern: when customers include the word “account” in their message (e.g., “I want to check my account for an order I made yesterday”), the agent calls get_customer first 78% of the time. When customers phrase similar requests without “account” (e.g., “I want to check an order I made yesterday”), it calls lookup_order first 93% of the time. Tool descriptions are clear and unambiguous. What is the most likely root cause of this discrepancy?

Question

What is the most likely root cause?

Options

A) The system prompt contains keyword-sensitive instructions that steer behavior based on terms like “account,” creating unintended tool-selection patterns. (Correct)
B) The model’s base training creates associations between “account” terminology and customer-related operations that override tool descriptions.
C) The model needs more training data on multi-concept messages and should be fine-tuned on examples containing both account and order terminology.
D) Tool descriptions need additional negative examples specifying when NOT to use each tool to prevent this keyword-induced confusion.

Explanation (Türkçe)

Sistemli anahtar kelime odaklı model (%78’e karşı %93), system prompt’undaki “account” kelimesine tepki veren ve agent’ı müşteriyle ilgili tool’lara yönlendiren açık bir yönlendirme mantığına (routing logic) işaret etmektedir. Tool açıklamaları zaten net olduğundan, bu sapma prompt düzeyindeki talimatların istenmeyen davranışsal yönlendirmelere (behavioral steering) neden olduğunu göstermektedir.

Q57 (Scenario: Customer Support Agent)

Situation

Production logs show the agent often calls get_customer when users ask about orders (e.g., “check my order #12345”) instead of calling lookup_order. Both tools have minimal descriptions (“Gets customer information” / “Gets order details”) and accept similar-looking identifier formats. What is the most effective first step to improve tool selection reliability?

Question

What is the most effective first step?

Options

A) Implement a routing layer that analyzes user input before each turn and preselects the correct tool based on detected keywords and ID patterns.
B) Combine both tools into a single lookup_entity that accepts any identifier and internally decides which backend to query.
C) Add few-shot examples to the system prompt demonstrating correct tool selection patterns, with 5–8 examples routing order-related queries to lookup_order.
D) Expand each tool’s description to include input formats, example queries, edge cases, and boundaries explaining when to use it versus similar tools. (Correct)

Explanation (Türkçe)

Tool açıklamalarını girdi formatları, örnek sorgular, edge case’ler ve net sınırlar içerecek şekilde genişletmek, LLM’e benzer tool’ları ayırt etmesi için yeterli bilgi sağlamayan minimal açıklamalar şeklindeki kök nedeni doğrudan düzeltir. Bu, LLM’in tool seçimi için kullandığı birincil mekanizmayı geliştiren düşük eforlu ve yüksek etkili bir ilk adımdır.

Q58 (Scenario: Customer Support Agent)

Situation

You are implementing the agent loop for your support agent. After each Claude API call, you must decide whether to continue the loop (run requested tools and call Claude again) or stop (present the final answer to the customer). What determines this decision?

Question

What determines this decision?

Options

A) Check the stop_reason field in Claude’s response—continue if it is tool_use and stop if it is end_turn. (Correct)
B) Parse Claude’s text for phrases like “I’m done” or “Can I help with anything else?”—natural language signals indicate task completion.
C) Set a maximum iteration count (e.g., 10 calls) and stop when reached, regardless of whether Claude indicates more work is needed.
D) Check whether the response contains assistant text content—if Claude generated explanatory text, the loop should terminate.

Explanation (Türkçe)

stop_reason, Claude’un döngü kontrolü için yapılandırılmış (structured) açık sinyalidir: tool_use Claude’un bir tool çalıştırmak ve sonuçları geri almak istediğini belirtirken, end_turn Claude’un yanıtını tamamladığını ve döngünün sona ermesi gerektiğini gösterir.

Q59 (Scenario: Customer Support Agent)

Situation

Production logs show the agent misinterprets outputs from your MCP tools: Unix timestamps from get_customer, ISO 8601 dates from lookup_order, and numeric status codes (1=pending, 2=shipped). Some tools are third-party MCP servers you cannot modify. Which approach to data format normalization is most maintainable?

Question

Which approach is most maintainable?

Options

A) Use a PostToolUse hook to intercept tool outputs and apply formatting transformations before the agent processes them. (Correct)
B) Modify tools you control to return human-readable formats and create wrappers for third-party tools.
C) Create a normalize_data tool that the agent calls after every data retrieval to transform values.
D) Add detailed format documentation to the system prompt explaining each tool’s data conventions.

Explanation (Türkçe)

Bir PostToolUse kancası (hook), agent verileri işlemeden önce üçüncü taraf MCP sunucu verileri de dahil olmak üzere tüm tool çıktılarını yakalamak (intercept) ve normalize etmek için merkezi ve deterministik bir nokta sağlar. Dönüşümler kod üzerinde yaşadığı ve LLM yorumuna güvenmek yerine tek biçimde uygulandığı için daha sürdürülebilirdir (maintainable).

Q60 (Scenario: Customer Support Agent)

Situation

Production logs show the agent sometimes chooses get_customer when lookup_order would be more appropriate, especially for ambiguous queries like “I need help with my recent purchase.” You decide to add few-shot examples to the system prompt to improve tool selection. Which approach most effectively addresses the problem?

Question

Which approach is most effective?

Options

A) Add explicit “use when” and “don’t use when” guidance in each tool description covering ambiguous cases.
B) Add examples grouped by tool—all get_customer scenarios together, then all lookup_order scenarios.
C) Add 4–6 examples targeted at ambiguous scenarios, each with rationale for why one tool was chosen over plausible alternatives. (Correct)
D) Add 10–15 examples of clear, unambiguous requests demonstrating correct tool choice for typical scenarios for each tool.

Options

A) Make reasonable assumptions, state them explicitly, and offer to adjust. (Correct)
B) Classify ambiguity with a smaller model before responding.
C) Use predefined interpretations without stating assumptions.
D) Limit the assistant to one clarifying question per turn.

Explanation (Türkçe)

Makul varsayımlarla ilerleyip bunları açıkça belirtmek ve ayar teklif etmek, gidiş-gelişleri tamamen ortadan kaldırırken kullanıcıyı bilgilendirilmiş ve kontrolde tutar. Sessiz, önceden tanımlı yorumlar (C) yanıt niyetle örtüşmediğinde kafa karıştırır. Tek-soru limiti (D) yine turlarca gidiş-geliş gerektirir. Küçük sınıflandırma modeli (B) çekirdek UX problemini çözmeden gecikme ve altyapı karmaşıklığı ekler.

Özet (sınav)

A/B/C/D dağılımı: Q1–Q60 tam dengeli (15’er); Q61–Q76 ile birlikte toplam 20 A, 17 B, 21 C, 18 D
Domain dağılımı: Q1–Q60’ta D1 (Agent architecture) 18, D2 (Tool/MCP) 10, D3 (Claude Code config) 16, D4 (Prompt/output) 10, D5 (Context/reliability) 18; Q61–Q76 ağırlıklı olarak D4 + D5 (stateless API, context yönetimi, system prompt drift, prefill, belirsizlik yönetimi)
Kritik kavramlar test edildi: escalation tetikleyicileri, attribution loss, structured error context, field-level confidence, hub-and-spoke, subagent delegation, self-review limitasyonu, lost-in-the-middle, summarization kayıp matrisi, programmatic preconditions, JSON schema validation, prompt caching, surgical changes, simplicity first
Zorluk: Orta-ileri. Gerçek sınav senaryolarının tipik güçlük seviyesi; her soruda “doğru cevap + 2-3 cümle gerekçe” var.
Zaman yönetimi: Gerçek sınav 90 dakika / 60 soru = 1.5 dk/soru. Bu 76 soruluk simülasyonu ~115 dakikada bitirmeyi hedefle. İlk 30 soruyu 45 dakikada bitir; takılırsan işaretleyip geç, sona dönüp tamamla.