feat: add diagnostic interview loop
This commit is contained in:
@@ -18,12 +18,12 @@ interview-ready after each short practice loop.
|
|||||||
|
|
||||||
### Interview Practice
|
### Interview Practice
|
||||||
|
|
||||||
- [ ] **INT-01**: User can select target role, stack, and interview timeline.
|
- [x] **INT-01**: User can select target role, stack, and interview timeline.
|
||||||
- [ ] **INT-02**: User can complete a diagnostic technical interview.
|
- [x] **INT-02**: User can complete a diagnostic technical interview.
|
||||||
- [ ] **INT-03**: System can generate role-specific interview questions.
|
- [x] **INT-03**: System can generate role-specific interview questions.
|
||||||
- [ ] **INT-04**: System can grade user answers against explicit rubrics.
|
- [x] **INT-04**: System can grade user answers against explicit rubrics.
|
||||||
- [ ] **INT-05**: System can ask targeted follow-up questions for weak answers.
|
- [x] **INT-05**: System can ask targeted follow-up questions for weak answers.
|
||||||
- [ ] **INT-06**: System preserves original answers and grading evidence.
|
- [x] **INT-06**: System preserves original answers and grading evidence.
|
||||||
|
|
||||||
### Learner Memory
|
### Learner Memory
|
||||||
|
|
||||||
@@ -95,7 +95,7 @@ interview-ready after each short practice loop.
|
|||||||
| Requirement | Phase | Status |
|
| Requirement | Phase | Status |
|
||||||
|-------------|-------|--------|
|
|-------------|-------|--------|
|
||||||
| BACK-01..BACK-05 | Phase 1 | Complete |
|
| BACK-01..BACK-05 | Phase 1 | Complete |
|
||||||
| INT-01..INT-06 | Phase 2 | Pending |
|
| INT-01..INT-06 | Phase 2 | Complete |
|
||||||
| MEM-01..MEM-05 | Phase 3 | Pending |
|
| MEM-01..MEM-05 | Phase 3 | Pending |
|
||||||
| PROG-01..PROG-05 | Phase 4 | Pending |
|
| PROG-01..PROG-05 | Phase 4 | Pending |
|
||||||
| ONTO-01..ONTO-04 | Phase 5 | Pending |
|
| ONTO-01..ONTO-04 | Phase 5 | Pending |
|
||||||
@@ -108,4 +108,4 @@ interview-ready after each short practice loop.
|
|||||||
|
|
||||||
---
|
---
|
||||||
*Requirements defined: 2026-04-26*
|
*Requirements defined: 2026-04-26*
|
||||||
*Last updated: 2026-04-26 after Phase 1 execution.*
|
*Last updated: 2026-04-26 after Phase 2 execution.*
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ See: `.planning/PROJECT.md` (updated 2026-04-26)
|
|||||||
**Core value:** The user should feel and prove that they are becoming more
|
**Core value:** The user should feel and prove that they are becoming more
|
||||||
interview-ready after each short practice loop.
|
interview-ready after each short practice loop.
|
||||||
|
|
||||||
**Current focus:** Phase 2 planning: Diagnostic Interview Loop.
|
**Current focus:** Phase 3 planning: Learner Memory.
|
||||||
|
|
||||||
## Current Decisions
|
## Current Decisions
|
||||||
|
|
||||||
@@ -22,14 +22,16 @@ interview-ready after each short practice loop.
|
|||||||
- First interview track is Backend Developer Interview.
|
- First interview track is Backend Developer Interview.
|
||||||
- Phase 1 has context, research, and plan artifacts.
|
- Phase 1 has context, research, and plan artifacts.
|
||||||
- Phase 1 Go backend scaffold is implemented and verified.
|
- Phase 1 Go backend scaffold is implemented and verified.
|
||||||
|
- Phase 2 diagnostic interview loop is implemented and verified with in-memory
|
||||||
|
sessions.
|
||||||
|
|
||||||
## Next Actions
|
## Next Actions
|
||||||
|
|
||||||
1. Plan Phase 2 diagnostic interview loop with GSD.
|
1. Plan Phase 3 learner memory with GSD.
|
||||||
2. Keep `docs/planning/WORKFLOW_CONTRACTS.md` aligned with Go structs during
|
2. Keep `docs/planning/WORKFLOW_CONTRACTS.md` aligned with Go structs during
|
||||||
future workflow implementation.
|
future workflow implementation.
|
||||||
3. Decide whether Phase 2 starts with in-memory diagnostic sessions or a small
|
3. Decide whether Phase 3 learner memory remains in-memory for MVP proof or
|
||||||
persistence boundary.
|
introduces a small persistence boundary.
|
||||||
|
|
||||||
## Validation Log
|
## Validation Log
|
||||||
|
|
||||||
@@ -40,6 +42,9 @@ interview-ready after each short practice loop.
|
|||||||
- 2026-04-26: Phase 1 implementation verified with `go test ./...`,
|
- 2026-04-26: Phase 1 implementation verified with `go test ./...`,
|
||||||
`openspec validate bootstrap-job-tutor-platform --strict`, and Go source
|
`openspec validate bootstrap-job-tutor-platform --strict`, and Go source
|
||||||
line-count check.
|
line-count check.
|
||||||
|
- 2026-04-26: Phase 2 implementation verified with `go test ./...`, live
|
||||||
|
`/healthz` smoke, live diagnostic create/answer/get smoke, OpenSpec, and Go
|
||||||
|
source line-count check.
|
||||||
|
|
||||||
---
|
---
|
||||||
*State initialized: 2026-04-26.*
|
*State initialized: 2026-04-26.*
|
||||||
|
|||||||
@@ -0,0 +1,93 @@
|
|||||||
|
# Phase 2: Diagnostic Interview Loop - Context
|
||||||
|
|
||||||
|
**Gathered:** 2026-04-26
|
||||||
|
**Status:** Ready for planning
|
||||||
|
**Source:** GSD continuation after Phase 1 completion
|
||||||
|
|
||||||
|
<domain>
|
||||||
|
## Phase Boundary
|
||||||
|
|
||||||
|
Phase 2 proves the first job-seeker loop from target role/stack selection
|
||||||
|
through a graded diagnostic answer. It should create a thin backend product
|
||||||
|
surface for diagnostic sessions while avoiding persistent storage and real LLM
|
||||||
|
calls until later phases require them.
|
||||||
|
|
||||||
|
</domain>
|
||||||
|
|
||||||
|
<decisions>
|
||||||
|
## Implementation Decisions
|
||||||
|
|
||||||
|
### Persistence
|
||||||
|
|
||||||
|
- Use an in-memory session store in Phase 2.
|
||||||
|
- Do not add a database or migrations yet.
|
||||||
|
- Persisting original answers and grading evidence means preserving them inside
|
||||||
|
the in-memory diagnostic session record for this phase.
|
||||||
|
|
||||||
|
### Interview Track
|
||||||
|
|
||||||
|
- Use the Backend Developer Interview track from
|
||||||
|
`docs/planning/INTERVIEW_TRACKS.md`.
|
||||||
|
- Seed questions should cover the canonical concept clusters, starting with a
|
||||||
|
small representative set.
|
||||||
|
|
||||||
|
### Workflow Boundary
|
||||||
|
|
||||||
|
- Grading must go through the typed `workflows.Runner` interface.
|
||||||
|
- The default implementation can remain deterministic/stubbed, but it must
|
||||||
|
return typed `GradedAnswer` data rather than freeform prose.
|
||||||
|
- HTTP handlers must not shell out or parse arbitrary assistant text.
|
||||||
|
|
||||||
|
</decisions>
|
||||||
|
|
||||||
|
<canonical_refs>
|
||||||
|
## Canonical References
|
||||||
|
|
||||||
|
Downstream agents MUST read these before planning or implementing.
|
||||||
|
|
||||||
|
### Product and Track
|
||||||
|
|
||||||
|
- `docs/planning/PRD.md` - diagnostic interview product flow.
|
||||||
|
- `docs/planning/INTERVIEW_TRACKS.md` - first track and concept seed list.
|
||||||
|
- `docs/planning/WORKFLOW_CONTRACTS.md` - typed workflow result shape.
|
||||||
|
|
||||||
|
### Engineering and Requirements
|
||||||
|
|
||||||
|
- `docs/planning/ENGINEERING.md` - 600-line, SOLID, KISS, YAGNI constraints.
|
||||||
|
- `.planning/REQUIREMENTS.md` - INT-01 through INT-06 requirements.
|
||||||
|
- `.planning/ROADMAP.md` - Phase 2 goal and success criteria.
|
||||||
|
- `openspec/changes/bootstrap-job-tutor-platform/specs/job-seeker-tutor/spec.md`
|
||||||
|
- diagnostic and first-track requirements.
|
||||||
|
- `openspec/changes/bootstrap-job-tutor-platform/specs/tutor-workflows/spec.md`
|
||||||
|
- typed workflow requirements.
|
||||||
|
|
||||||
|
</canonical_refs>
|
||||||
|
|
||||||
|
<specifics>
|
||||||
|
## Specific Ideas
|
||||||
|
|
||||||
|
- Add `internal/interview` for sessions, questions, answers, and in-memory store.
|
||||||
|
- Add endpoints:
|
||||||
|
- `POST /api/v1/diagnostic-sessions`
|
||||||
|
- `GET /api/v1/diagnostic-sessions/{id}`
|
||||||
|
- `POST /api/v1/diagnostic-sessions/{id}/answers`
|
||||||
|
- Keep routing simple with standard library `http.ServeMux`.
|
||||||
|
- Add deterministic grading in the workflow stub so tests can prove typed
|
||||||
|
evidence is recorded.
|
||||||
|
|
||||||
|
</specifics>
|
||||||
|
|
||||||
|
<deferred>
|
||||||
|
## Deferred Ideas
|
||||||
|
|
||||||
|
- Real `third-one` grading execution.
|
||||||
|
- Database persistence.
|
||||||
|
- Authentication and user identity provider integration.
|
||||||
|
- Memory extraction and durable learner memory.
|
||||||
|
- Frontend UI.
|
||||||
|
|
||||||
|
</deferred>
|
||||||
|
|
||||||
|
---
|
||||||
|
*Phase: 002-diagnostic-interview-loop*
|
||||||
|
*Context gathered: 2026-04-26*
|
||||||
77
.planning/phases/002-diagnostic-interview-loop/002-PLAN.md
Normal file
77
.planning/phases/002-diagnostic-interview-loop/002-PLAN.md
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
# Phase 2 Plan: Diagnostic Interview Loop
|
||||||
|
|
||||||
|
**Status:** Ready for execution
|
||||||
|
**Phase Goal:** Prove the first job-seeker loop from role selection through
|
||||||
|
graded diagnostic interview.
|
||||||
|
|
||||||
|
## Requirements Covered
|
||||||
|
|
||||||
|
- INT-01: User can select target role, stack, and interview timeline.
|
||||||
|
- INT-02: User can complete a diagnostic technical interview.
|
||||||
|
- INT-03: System can generate role-specific interview questions.
|
||||||
|
- INT-04: System can grade user answers against explicit rubrics.
|
||||||
|
- INT-05: System can ask targeted follow-up questions for weak answers.
|
||||||
|
- INT-06: System preserves original answers and grading evidence.
|
||||||
|
|
||||||
|
## Tasks
|
||||||
|
|
||||||
|
### 1. Add interview domain package
|
||||||
|
|
||||||
|
- Create `internal/interview`.
|
||||||
|
- Define session, question, answer, and store types.
|
||||||
|
- Add in-memory store implementation.
|
||||||
|
- Add backend developer diagnostic question catalog.
|
||||||
|
|
||||||
|
### 2. Add diagnostic service
|
||||||
|
|
||||||
|
- Create session from target role, stack, and optional interview timeline.
|
||||||
|
- Select first diagnostic questions from the backend developer track.
|
||||||
|
- Record submitted answers.
|
||||||
|
- Invoke `workflows.Runner.GradeInterviewAnswer`.
|
||||||
|
- Attach typed grading result and evidence to the answer record.
|
||||||
|
|
||||||
|
### 3. Add HTTP endpoints
|
||||||
|
|
||||||
|
- `POST /api/v1/diagnostic-sessions`
|
||||||
|
- `GET /api/v1/diagnostic-sessions/{id}`
|
||||||
|
- `POST /api/v1/diagnostic-sessions/{id}/answers`
|
||||||
|
|
||||||
|
### 4. Extend workflow stub
|
||||||
|
|
||||||
|
- Return deterministic typed grades.
|
||||||
|
- Include follow-up recommendation for weak answers.
|
||||||
|
- Include evidence references.
|
||||||
|
|
||||||
|
### 5. Add tests
|
||||||
|
|
||||||
|
- Domain tests for session creation and answer grading.
|
||||||
|
- HTTP tests for create/get/answer flow.
|
||||||
|
- Existing config/workflow tests remain passing.
|
||||||
|
|
||||||
|
### 6. Update GSD/OpenSpec state
|
||||||
|
|
||||||
|
- Mark INT-01 through INT-06 complete if all success criteria pass.
|
||||||
|
- Add Phase 2 summary and verification artifacts.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
gofmt -w cmd internal
|
||||||
|
go test ./...
|
||||||
|
openspec validate bootstrap-job-tutor-platform --strict
|
||||||
|
```
|
||||||
|
|
||||||
|
Run Go line-count check and confirm every manually authored Go file is at or
|
||||||
|
below 600 lines.
|
||||||
|
|
||||||
|
## Out of Scope
|
||||||
|
|
||||||
|
- Database persistence.
|
||||||
|
- Authentication.
|
||||||
|
- Real LLM grading.
|
||||||
|
- Durable learner memory.
|
||||||
|
- Progression map.
|
||||||
|
- Frontend.
|
||||||
|
|
||||||
|
---
|
||||||
|
*Plan created: 2026-04-26*
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
# Phase 2 Research
|
||||||
|
|
||||||
|
## Question
|
||||||
|
|
||||||
|
How should the diagnostic interview loop be implemented while preserving the
|
||||||
|
Phase 1 typed workflow boundary and avoiding premature infrastructure?
|
||||||
|
|
||||||
|
## Findings
|
||||||
|
|
||||||
|
### In-memory persistence is enough for Phase 2
|
||||||
|
|
||||||
|
The goal is to prove request/response flow and evidence preservation. A database
|
||||||
|
would add migration, repository, and lifecycle complexity before the product loop
|
||||||
|
is proven. An in-memory store with clear interface boundaries keeps the future
|
||||||
|
database replacement straightforward.
|
||||||
|
|
||||||
|
### Standard library routing remains sufficient
|
||||||
|
|
||||||
|
The current backend already uses `http.ServeMux`. Phase 2 can add route patterns
|
||||||
|
with path variables using Go 1.23's standard mux support. No router dependency
|
||||||
|
is needed.
|
||||||
|
|
||||||
|
### Deterministic grading is acceptable as a workflow stub
|
||||||
|
|
||||||
|
Phase 2 requires typed grading through the workflow boundary. It does not require
|
||||||
|
live LLM grading. A deterministic stub can grade on answer length and preserve
|
||||||
|
evidence. This proves the product state flow and keeps live model integration
|
||||||
|
for a later phase.
|
||||||
|
|
||||||
|
### Keep interview domain separate from HTTP
|
||||||
|
|
||||||
|
`internal/interview` should own session creation, question catalog selection,
|
||||||
|
answer recording, and grade attachment. HTTP handlers should translate requests
|
||||||
|
and responses only.
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
|
||||||
|
1. Add `internal/interview` domain service and in-memory store.
|
||||||
|
2. Add a small backend developer question catalog.
|
||||||
|
3. Add typed endpoints for creating/getting sessions and submitting answers.
|
||||||
|
4. Extend workflow stub to return deterministic `GradedAnswer`.
|
||||||
|
5. Add tests at domain and HTTP layers.
|
||||||
|
6. Verify with `go test ./...`, OpenSpec, and line-count checks.
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# Phase 2 Summary
|
||||||
|
|
||||||
|
**Status:** Complete
|
||||||
|
**Completed:** 2026-04-26
|
||||||
|
|
||||||
|
## Delivered
|
||||||
|
|
||||||
|
- Added in-memory diagnostic interview domain package.
|
||||||
|
- Added Backend Developer Interview seed question catalog.
|
||||||
|
- Added diagnostic session create/get/answer service.
|
||||||
|
- Added session status that becomes `complete` after all diagnostic questions
|
||||||
|
have answers.
|
||||||
|
- Added HTTP endpoints:
|
||||||
|
- `POST /api/v1/diagnostic-sessions`
|
||||||
|
- `GET /api/v1/diagnostic-sessions/{id}`
|
||||||
|
- `POST /api/v1/diagnostic-sessions/{id}/answers`
|
||||||
|
- Extended workflow stub to return deterministic typed grading results.
|
||||||
|
- Added grading evidence to `GradedAnswer`.
|
||||||
|
- Added domain, workflow, and HTTP flow tests.
|
||||||
|
|
||||||
|
## Files Added
|
||||||
|
|
||||||
|
- `internal/httpapi/diagnostic.go`
|
||||||
|
- `internal/httpapi/diagnostic_test.go`
|
||||||
|
- `internal/interview/catalog.go`
|
||||||
|
- `internal/interview/service.go`
|
||||||
|
- `internal/interview/service_test.go`
|
||||||
|
- `internal/interview/store.go`
|
||||||
|
- `internal/interview/types.go`
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
gofmt -w cmd internal
|
||||||
|
go test ./...
|
||||||
|
openspec validate bootstrap-job-tutor-platform --strict
|
||||||
|
```
|
||||||
|
|
||||||
|
Additional live smoke checks:
|
||||||
|
|
||||||
|
- `GET /healthz` returned status `ok`.
|
||||||
|
- Diagnostic create/answer/get flow returned a session id, 3 questions, a
|
||||||
|
`solid` typed grade, 1 evidence item, and 1 stored answer.
|
||||||
|
|
||||||
|
## Deferred
|
||||||
|
|
||||||
|
- Durable database persistence.
|
||||||
|
- Authentication.
|
||||||
|
- Real `third-one` grading calls.
|
||||||
|
- Learner memory extraction and readiness progression.
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# Phase 2 Verification
|
||||||
|
|
||||||
|
## Verdict
|
||||||
|
|
||||||
|
PASS
|
||||||
|
|
||||||
|
## Requirement Coverage
|
||||||
|
|
||||||
|
- INT-01: PASS. Diagnostic session request accepts target role, stack, and
|
||||||
|
interview timeline.
|
||||||
|
- INT-02: PASS. Diagnostic sessions can progress to `complete` after all seed
|
||||||
|
questions are answered.
|
||||||
|
- INT-03: PASS. Backend Developer Interview questions are generated from the
|
||||||
|
role-specific seed catalog.
|
||||||
|
- INT-04: PASS. Answers are graded through the typed workflow runner boundary.
|
||||||
|
- INT-05: PASS. Weak or partial answers receive typed follow-up recommendations.
|
||||||
|
- INT-06: PASS. Original answer text and grading evidence are preserved in the
|
||||||
|
in-memory session record.
|
||||||
|
|
||||||
|
## Evidence
|
||||||
|
|
||||||
|
- `go test ./...` passed.
|
||||||
|
- `openspec validate bootstrap-job-tutor-platform --strict` passed.
|
||||||
|
- Live `GET /healthz` smoke passed.
|
||||||
|
- Live diagnostic create/answer/get smoke passed.
|
||||||
|
- Go source line-count check passed.
|
||||||
|
|
||||||
|
## Residual Risk
|
||||||
|
|
||||||
|
Persistence is intentionally in-memory. Data is lost on process restart. Phase 3
|
||||||
|
should decide whether learner memory remains in-memory for proof or introduces a
|
||||||
|
small persistent boundary.
|
||||||
@@ -73,6 +73,7 @@ Produced by `grade_interview_answer`.
|
|||||||
"overall": "miss|partial|solid|strong",
|
"overall": "miss|partial|solid|strong",
|
||||||
"strengths": ["string"],
|
"strengths": ["string"],
|
||||||
"gaps": ["string"],
|
"gaps": ["string"],
|
||||||
|
"evidence": ["evidence_ref"],
|
||||||
"misconception_candidates": [
|
"misconception_candidates": [
|
||||||
{
|
{
|
||||||
"label": "string",
|
"label": "string",
|
||||||
|
|||||||
@@ -5,12 +5,15 @@ import (
|
|||||||
|
|
||||||
"tutor/internal/config"
|
"tutor/internal/config"
|
||||||
"tutor/internal/httpapi"
|
"tutor/internal/httpapi"
|
||||||
|
"tutor/internal/interview"
|
||||||
"tutor/internal/workflows"
|
"tutor/internal/workflows"
|
||||||
)
|
)
|
||||||
|
|
||||||
func NewServer(cfg config.Config) *http.Server {
|
func NewServer(cfg config.Config) *http.Server {
|
||||||
runner := workflows.NewStubRunner()
|
runner := workflows.NewStubRunner()
|
||||||
handler := httpapi.NewHandler(cfg, runner)
|
store := interview.NewMemoryStore()
|
||||||
|
service := interview.NewService(store, runner)
|
||||||
|
handler := httpapi.NewHandler(cfg, service)
|
||||||
|
|
||||||
return &http.Server{
|
return &http.Server{
|
||||||
Addr: cfg.HTTPAddr,
|
Addr: cfg.HTTPAddr,
|
||||||
|
|||||||
80
internal/httpapi/diagnostic.go
Normal file
80
internal/httpapi/diagnostic.go
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
package httpapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"tutor/internal/interview"
|
||||||
|
)
|
||||||
|
|
||||||
|
type createDiagnosticSessionRequest struct {
|
||||||
|
UserID string `json:"user_id"`
|
||||||
|
TargetRole string `json:"target_role"`
|
||||||
|
Stack []string `json:"stack"`
|
||||||
|
InterviewTimeline string `json:"interview_timeline"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type submitDiagnosticAnswerRequest struct {
|
||||||
|
QuestionID string `json:"question_id"`
|
||||||
|
AnswerText string `json:"answer_text"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h Handler) createDiagnosticSession(w http.ResponseWriter, r *http.Request) {
|
||||||
|
var req createDiagnosticSessionRequest
|
||||||
|
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||||
|
writeError(w, http.StatusBadRequest, "invalid JSON body")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
session, err := h.diagnostic.CreateSession(r.Context(), interview.CreateSessionInput{
|
||||||
|
UserID: req.UserID,
|
||||||
|
TargetRole: req.TargetRole,
|
||||||
|
Stack: req.Stack,
|
||||||
|
InterviewTimeline: req.InterviewTimeline,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
writeError(w, http.StatusBadRequest, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
writeJSON(w, http.StatusCreated, session)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h Handler) getDiagnosticSession(w http.ResponseWriter, r *http.Request) {
|
||||||
|
session, err := h.diagnostic.GetSession(r.PathValue("id"))
|
||||||
|
if errors.Is(err, interview.ErrSessionNotFound) {
|
||||||
|
writeError(w, http.StatusNotFound, "diagnostic session not found")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
writeError(w, http.StatusInternalServerError, "could not load diagnostic session")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
writeJSON(w, http.StatusOK, session)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h Handler) submitDiagnosticAnswer(w http.ResponseWriter, r *http.Request) {
|
||||||
|
var req submitDiagnosticAnswerRequest
|
||||||
|
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||||
|
writeError(w, http.StatusBadRequest, "invalid JSON body")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
answer, err := h.diagnostic.SubmitAnswer(r.Context(), interview.SubmitAnswerInput{
|
||||||
|
SessionID: r.PathValue("id"),
|
||||||
|
QuestionID: req.QuestionID,
|
||||||
|
AnswerText: req.AnswerText,
|
||||||
|
})
|
||||||
|
if errors.Is(err, interview.ErrSessionNotFound) || errors.Is(err, interview.ErrQuestionNotFound) {
|
||||||
|
writeError(w, http.StatusNotFound, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
writeError(w, http.StatusBadRequest, err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
writeJSON(w, http.StatusCreated, answer)
|
||||||
|
}
|
||||||
76
internal/httpapi/diagnostic_test.go
Normal file
76
internal/httpapi/diagnostic_test.go
Normal file
@@ -0,0 +1,76 @@
|
|||||||
|
package httpapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"tutor/internal/config"
|
||||||
|
"tutor/internal/interview"
|
||||||
|
"tutor/internal/workflows"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDiagnosticHTTPFlow(t *testing.T) {
|
||||||
|
service := interview.NewService(interview.NewMemoryStore(), workflows.NewStubRunner())
|
||||||
|
handler := NewHandler(config.Config{Environment: "test", ModelKey: "deepseek-v4-flash"}, service)
|
||||||
|
routes := handler.Routes()
|
||||||
|
|
||||||
|
createBody := bytes.NewBufferString(`{
|
||||||
|
"user_id":"user-1",
|
||||||
|
"target_role":"junior backend developer",
|
||||||
|
"stack":["go","postgres"],
|
||||||
|
"interview_timeline":"30 days"
|
||||||
|
}`)
|
||||||
|
createReq := httptest.NewRequest(http.MethodPost, "/api/v1/diagnostic-sessions", createBody)
|
||||||
|
createRec := httptest.NewRecorder()
|
||||||
|
routes.ServeHTTP(createRec, createReq)
|
||||||
|
|
||||||
|
if createRec.Code != http.StatusCreated {
|
||||||
|
t.Fatalf("create status = %d, body = %s", createRec.Code, createRec.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
var session interview.Session
|
||||||
|
if err := json.NewDecoder(createRec.Body).Decode(&session); err != nil {
|
||||||
|
t.Fatalf("decode create response: %v", err)
|
||||||
|
}
|
||||||
|
if len(session.Questions) == 0 {
|
||||||
|
t.Fatal("expected questions")
|
||||||
|
}
|
||||||
|
|
||||||
|
answerBody := bytes.NewBufferString(`{
|
||||||
|
"question_id":"` + session.Questions[0].ID + `",
|
||||||
|
"answer_text":"Idempotent requests can be retried safely because repeated calls have the same intended effect."
|
||||||
|
}`)
|
||||||
|
answerReq := httptest.NewRequest(http.MethodPost, "/api/v1/diagnostic-sessions/"+session.ID+"/answers", answerBody)
|
||||||
|
answerRec := httptest.NewRecorder()
|
||||||
|
routes.ServeHTTP(answerRec, answerReq)
|
||||||
|
|
||||||
|
if answerRec.Code != http.StatusCreated {
|
||||||
|
t.Fatalf("answer status = %d, body = %s", answerRec.Code, answerRec.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
var answer interview.Answer
|
||||||
|
if err := json.NewDecoder(answerRec.Body).Decode(&answer); err != nil {
|
||||||
|
t.Fatalf("decode answer response: %v", err)
|
||||||
|
}
|
||||||
|
if len(answer.Grade.Evidence) == 0 {
|
||||||
|
t.Fatal("expected grade evidence")
|
||||||
|
}
|
||||||
|
|
||||||
|
getReq := httptest.NewRequest(http.MethodGet, "/api/v1/diagnostic-sessions/"+session.ID, nil)
|
||||||
|
getRec := httptest.NewRecorder()
|
||||||
|
routes.ServeHTTP(getRec, getReq)
|
||||||
|
|
||||||
|
if getRec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("get status = %d, body = %s", getRec.Code, getRec.Body.String())
|
||||||
|
}
|
||||||
|
var loaded interview.Session
|
||||||
|
if err := json.NewDecoder(getRec.Body).Decode(&loaded); err != nil {
|
||||||
|
t.Fatalf("decode get response: %v", err)
|
||||||
|
}
|
||||||
|
if len(loaded.Answers) != 1 {
|
||||||
|
t.Fatalf("answers = %d, want 1", len(loaded.Answers))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -5,24 +5,27 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
|
|
||||||
"tutor/internal/config"
|
"tutor/internal/config"
|
||||||
"tutor/internal/workflows"
|
"tutor/internal/interview"
|
||||||
)
|
)
|
||||||
|
|
||||||
type Handler struct {
|
type Handler struct {
|
||||||
cfg config.Config
|
cfg config.Config
|
||||||
runner workflows.Runner
|
diagnostic *interview.Service
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewHandler(cfg config.Config, runner workflows.Runner) Handler {
|
func NewHandler(cfg config.Config, diagnostic *interview.Service) Handler {
|
||||||
return Handler{
|
return Handler{
|
||||||
cfg: cfg,
|
cfg: cfg,
|
||||||
runner: runner,
|
diagnostic: diagnostic,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (h Handler) Routes() http.Handler {
|
func (h Handler) Routes() http.Handler {
|
||||||
mux := http.NewServeMux()
|
mux := http.NewServeMux()
|
||||||
mux.HandleFunc("GET /healthz", h.health)
|
mux.HandleFunc("GET /healthz", h.health)
|
||||||
|
mux.HandleFunc("POST /api/v1/diagnostic-sessions", h.createDiagnosticSession)
|
||||||
|
mux.HandleFunc("GET /api/v1/diagnostic-sessions/{id}", h.getDiagnosticSession)
|
||||||
|
mux.HandleFunc("POST /api/v1/diagnostic-sessions/{id}/answers", h.submitDiagnosticAnswer)
|
||||||
return mux
|
return mux
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -45,3 +48,11 @@ func writeJSON(w http.ResponseWriter, status int, value any) {
|
|||||||
w.WriteHeader(status)
|
w.WriteHeader(status)
|
||||||
_ = json.NewEncoder(w).Encode(value)
|
_ = json.NewEncoder(w).Encode(value)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func writeError(w http.ResponseWriter, status int, message string) {
|
||||||
|
writeJSON(w, status, errorResponse{Error: message})
|
||||||
|
}
|
||||||
|
|
||||||
|
type errorResponse struct {
|
||||||
|
Error string `json:"error"`
|
||||||
|
}
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"tutor/internal/config"
|
"tutor/internal/config"
|
||||||
|
"tutor/internal/interview"
|
||||||
"tutor/internal/workflows"
|
"tutor/internal/workflows"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -15,7 +16,8 @@ func TestHealth(t *testing.T) {
|
|||||||
Environment: "test",
|
Environment: "test",
|
||||||
ModelKey: "deepseek-v4-flash",
|
ModelKey: "deepseek-v4-flash",
|
||||||
}
|
}
|
||||||
handler := NewHandler(cfg, workflows.NewStubRunner())
|
service := interview.NewService(interview.NewMemoryStore(), workflows.NewStubRunner())
|
||||||
|
handler := NewHandler(cfg, service)
|
||||||
|
|
||||||
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
|
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
|
||||||
rec := httptest.NewRecorder()
|
rec := httptest.NewRecorder()
|
||||||
|
|||||||
29
internal/interview/catalog.go
Normal file
29
internal/interview/catalog.go
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
package interview
|
||||||
|
|
||||||
|
import "tutor/internal/workflows"
|
||||||
|
|
||||||
|
func BackendDeveloperQuestions() []Question {
|
||||||
|
return []Question{
|
||||||
|
{
|
||||||
|
ID: "backend-http-idempotency",
|
||||||
|
Prompt: "What makes an HTTP method idempotent, and why does that matter for retries?",
|
||||||
|
Concepts: []workflows.ConceptRef{
|
||||||
|
{ID: "http-idempotency", Label: "HTTP idempotency", Track: BackendDeveloperTrack},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
ID: "backend-db-index-tradeoff",
|
||||||
|
Prompt: "When would adding a database index improve an API, and what tradeoffs can it introduce?",
|
||||||
|
Concepts: []workflows.ConceptRef{
|
||||||
|
{ID: "database-indexes", Label: "Database indexes", Track: BackendDeveloperTrack},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
ID: "backend-cache-invalidation",
|
||||||
|
Prompt: "How would you decide whether to cache an API response, and how would you handle stale data?",
|
||||||
|
Concepts: []workflows.ConceptRef{
|
||||||
|
{ID: "cache-invalidation", Label: "Cache invalidation", Track: BackendDeveloperTrack},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
117
internal/interview/service.go
Normal file
117
internal/interview/service.go
Normal file
@@ -0,0 +1,117 @@
|
|||||||
|
package interview
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"sync/atomic"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"tutor/internal/workflows"
|
||||||
|
)
|
||||||
|
|
||||||
|
var ErrQuestionNotFound = errors.New("diagnostic question not found")
|
||||||
|
|
||||||
|
type Service struct {
|
||||||
|
store Store
|
||||||
|
runner workflows.Runner
|
||||||
|
ids atomic.Uint64
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewService(store Store, runner workflows.Runner) *Service {
|
||||||
|
return &Service{store: store, runner: runner}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Service) CreateSession(_ context.Context, input CreateSessionInput) (Session, error) {
|
||||||
|
if strings.TrimSpace(input.UserID) == "" {
|
||||||
|
return Session{}, errors.New("user_id is required")
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(input.TargetRole) == "" {
|
||||||
|
return Session{}, errors.New("target_role is required")
|
||||||
|
}
|
||||||
|
if len(input.Stack) == 0 {
|
||||||
|
return Session{}, errors.New("stack is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
session := Session{
|
||||||
|
ID: s.nextID("diag"),
|
||||||
|
UserID: input.UserID,
|
||||||
|
Track: BackendDeveloperTrack,
|
||||||
|
Status: SessionInProgress,
|
||||||
|
TargetRole: input.TargetRole,
|
||||||
|
Stack: append([]string(nil), input.Stack...),
|
||||||
|
InterviewTimeline: input.InterviewTimeline,
|
||||||
|
Questions: BackendDeveloperQuestions(),
|
||||||
|
CreatedAt: time.Now().UTC(),
|
||||||
|
}
|
||||||
|
return s.store.Create(session)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Service) GetSession(id string) (Session, error) {
|
||||||
|
return s.store.Get(id)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Service) SubmitAnswer(ctx context.Context, input SubmitAnswerInput) (Answer, error) {
|
||||||
|
if strings.TrimSpace(input.AnswerText) == "" {
|
||||||
|
return Answer{}, errors.New("answer_text is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
session, err := s.store.Get(input.SessionID)
|
||||||
|
if err != nil {
|
||||||
|
return Answer{}, err
|
||||||
|
}
|
||||||
|
|
||||||
|
question, ok := findQuestion(session.Questions, input.QuestionID)
|
||||||
|
if !ok {
|
||||||
|
return Answer{}, ErrQuestionNotFound
|
||||||
|
}
|
||||||
|
|
||||||
|
answer := Answer{
|
||||||
|
ID: s.nextID("answer"),
|
||||||
|
QuestionID: input.QuestionID,
|
||||||
|
Text: input.AnswerText,
|
||||||
|
CreatedAt: time.Now().UTC(),
|
||||||
|
}
|
||||||
|
grade, err := s.runner.GradeInterviewAnswer(ctx, workflows.GradeAnswerInput{
|
||||||
|
UserID: session.UserID,
|
||||||
|
QuestionID: question.ID,
|
||||||
|
AnswerID: answer.ID,
|
||||||
|
AnswerText: answer.Text,
|
||||||
|
Concepts: question.Concepts,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return Answer{}, err
|
||||||
|
}
|
||||||
|
answer.Grade = grade
|
||||||
|
|
||||||
|
session.Answers = append(session.Answers, answer)
|
||||||
|
if answeredQuestionCount(session.Answers) >= len(session.Questions) {
|
||||||
|
session.Status = SessionComplete
|
||||||
|
}
|
||||||
|
if _, err := s.store.Update(session); err != nil {
|
||||||
|
return Answer{}, err
|
||||||
|
}
|
||||||
|
return answer, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func answeredQuestionCount(answers []Answer) int {
|
||||||
|
answered := make(map[string]struct{}, len(answers))
|
||||||
|
for _, answer := range answers {
|
||||||
|
answered[answer.QuestionID] = struct{}{}
|
||||||
|
}
|
||||||
|
return len(answered)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Service) nextID(prefix string) string {
|
||||||
|
return fmt.Sprintf("%s-%d", prefix, s.ids.Add(1))
|
||||||
|
}
|
||||||
|
|
||||||
|
func findQuestion(questions []Question, id string) (Question, bool) {
|
||||||
|
for _, question := range questions {
|
||||||
|
if question.ID == id {
|
||||||
|
return question, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return Question{}, false
|
||||||
|
}
|
||||||
87
internal/interview/service_test.go
Normal file
87
internal/interview/service_test.go
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
package interview
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"tutor/internal/workflows"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDiagnosticSessionAnswerFlow(t *testing.T) {
|
||||||
|
service := NewService(NewMemoryStore(), workflows.NewStubRunner())
|
||||||
|
|
||||||
|
session, err := service.CreateSession(context.Background(), CreateSessionInput{
|
||||||
|
UserID: "user-1",
|
||||||
|
TargetRole: "junior backend developer",
|
||||||
|
Stack: []string{"go", "postgres"},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CreateSession error: %v", err)
|
||||||
|
}
|
||||||
|
if session.Track != BackendDeveloperTrack {
|
||||||
|
t.Fatalf("Track = %q", session.Track)
|
||||||
|
}
|
||||||
|
if session.Status != SessionInProgress {
|
||||||
|
t.Fatalf("Status = %q, want %q", session.Status, SessionInProgress)
|
||||||
|
}
|
||||||
|
if len(session.Questions) == 0 {
|
||||||
|
t.Fatal("expected diagnostic questions")
|
||||||
|
}
|
||||||
|
|
||||||
|
answer, err := service.SubmitAnswer(context.Background(), SubmitAnswerInput{
|
||||||
|
SessionID: session.ID,
|
||||||
|
QuestionID: session.Questions[0].ID,
|
||||||
|
AnswerText: "Idempotent methods can be retried safely because repeated calls have the same intended effect.",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("SubmitAnswer error: %v", err)
|
||||||
|
}
|
||||||
|
if answer.Grade.AnswerID != answer.ID {
|
||||||
|
t.Fatalf("grade answer id = %q, want %q", answer.Grade.AnswerID, answer.ID)
|
||||||
|
}
|
||||||
|
if len(answer.Grade.Concepts) == 0 {
|
||||||
|
t.Fatal("expected graded concepts")
|
||||||
|
}
|
||||||
|
if len(answer.Grade.Evidence) == 0 {
|
||||||
|
t.Fatal("expected grading evidence")
|
||||||
|
}
|
||||||
|
|
||||||
|
loaded, err := service.GetSession(session.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetSession error: %v", err)
|
||||||
|
}
|
||||||
|
if len(loaded.Answers) != 1 {
|
||||||
|
t.Fatalf("answers = %d, want 1", len(loaded.Answers))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDiagnosticSessionCompletesAfterAllQuestionsAnswered(t *testing.T) {
|
||||||
|
service := NewService(NewMemoryStore(), workflows.NewStubRunner())
|
||||||
|
|
||||||
|
session, err := service.CreateSession(context.Background(), CreateSessionInput{
|
||||||
|
UserID: "user-1",
|
||||||
|
TargetRole: "junior backend developer",
|
||||||
|
Stack: []string{"go"},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CreateSession error: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, question := range session.Questions {
|
||||||
|
if _, err := service.SubmitAnswer(context.Background(), SubmitAnswerInput{
|
||||||
|
SessionID: session.ID,
|
||||||
|
QuestionID: question.ID,
|
||||||
|
AnswerText: "This answer gives a concrete backend tradeoff with an operational example for the interview.",
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("SubmitAnswer(%s) error: %v", question.ID, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
loaded, err := service.GetSession(session.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GetSession error: %v", err)
|
||||||
|
}
|
||||||
|
if loaded.Status != SessionComplete {
|
||||||
|
t.Fatalf("Status = %q, want %q", loaded.Status, SessionComplete)
|
||||||
|
}
|
||||||
|
}
|
||||||
62
internal/interview/store.go
Normal file
62
internal/interview/store.go
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
package interview
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
var ErrSessionNotFound = errors.New("diagnostic session not found")
|
||||||
|
|
||||||
|
type Store interface {
|
||||||
|
Create(Session) (Session, error)
|
||||||
|
Get(string) (Session, error)
|
||||||
|
Update(Session) (Session, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type MemoryStore struct {
|
||||||
|
mu sync.RWMutex
|
||||||
|
sessions map[string]Session
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewMemoryStore() *MemoryStore {
|
||||||
|
return &MemoryStore{
|
||||||
|
sessions: make(map[string]Session),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *MemoryStore) Create(session Session) (Session, error) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
|
||||||
|
s.sessions[session.ID] = cloneSession(session)
|
||||||
|
return session, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *MemoryStore) Get(id string) (Session, error) {
|
||||||
|
s.mu.RLock()
|
||||||
|
defer s.mu.RUnlock()
|
||||||
|
|
||||||
|
session, ok := s.sessions[id]
|
||||||
|
if !ok {
|
||||||
|
return Session{}, ErrSessionNotFound
|
||||||
|
}
|
||||||
|
return cloneSession(session), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *MemoryStore) Update(session Session) (Session, error) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
|
||||||
|
if _, ok := s.sessions[session.ID]; !ok {
|
||||||
|
return Session{}, ErrSessionNotFound
|
||||||
|
}
|
||||||
|
s.sessions[session.ID] = cloneSession(session)
|
||||||
|
return session, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func cloneSession(session Session) Session {
|
||||||
|
session.Stack = append([]string(nil), session.Stack...)
|
||||||
|
session.Questions = append([]Question(nil), session.Questions...)
|
||||||
|
session.Answers = append([]Answer(nil), session.Answers...)
|
||||||
|
return session
|
||||||
|
}
|
||||||
56
internal/interview/types.go
Normal file
56
internal/interview/types.go
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
package interview
|
||||||
|
|
||||||
|
import (
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"tutor/internal/workflows"
|
||||||
|
)
|
||||||
|
|
||||||
|
const BackendDeveloperTrack = "backend-developer"
|
||||||
|
|
||||||
|
type SessionStatus string
|
||||||
|
|
||||||
|
const (
|
||||||
|
SessionInProgress SessionStatus = "in_progress"
|
||||||
|
SessionComplete SessionStatus = "complete"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Session struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
UserID string `json:"user_id"`
|
||||||
|
Track string `json:"track"`
|
||||||
|
Status SessionStatus `json:"status"`
|
||||||
|
TargetRole string `json:"target_role"`
|
||||||
|
Stack []string `json:"stack"`
|
||||||
|
InterviewTimeline string `json:"interview_timeline,omitempty"`
|
||||||
|
Questions []Question `json:"questions"`
|
||||||
|
Answers []Answer `json:"answers"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Question struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Prompt string `json:"prompt"`
|
||||||
|
Concepts []workflows.ConceptRef `json:"concepts"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Answer struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
QuestionID string `json:"question_id"`
|
||||||
|
Text string `json:"text"`
|
||||||
|
Grade workflows.GradedAnswer `json:"grade"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type CreateSessionInput struct {
|
||||||
|
UserID string
|
||||||
|
TargetRole string
|
||||||
|
Stack []string
|
||||||
|
InterviewTimeline string
|
||||||
|
}
|
||||||
|
|
||||||
|
type SubmitAnswerInput struct {
|
||||||
|
SessionID string
|
||||||
|
QuestionID string
|
||||||
|
AnswerText string
|
||||||
|
}
|
||||||
@@ -58,6 +58,7 @@ type GradedAnswer struct {
|
|||||||
Overall AnswerOverall `json:"overall"`
|
Overall AnswerOverall `json:"overall"`
|
||||||
Strengths []string `json:"strengths"`
|
Strengths []string `json:"strengths"`
|
||||||
Gaps []string `json:"gaps"`
|
Gaps []string `json:"gaps"`
|
||||||
|
Evidence []EvidenceRef `json:"evidence"`
|
||||||
MisconceptionCandidates []MisconceptionCandidate `json:"misconception_candidates"`
|
MisconceptionCandidates []MisconceptionCandidate `json:"misconception_candidates"`
|
||||||
FollowUp FollowUpRecommendation `json:"follow_up"`
|
FollowUp FollowUpRecommendation `json:"follow_up"`
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package workflows
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
var ErrNotImplemented = errors.New("workflow runner not implemented")
|
var ErrNotImplemented = errors.New("workflow runner not implemented")
|
||||||
@@ -27,6 +28,7 @@ type GradeAnswerInput struct {
|
|||||||
QuestionID string
|
QuestionID string
|
||||||
AnswerID string
|
AnswerID string
|
||||||
AnswerText string
|
AnswerText string
|
||||||
|
Concepts []ConceptRef
|
||||||
}
|
}
|
||||||
|
|
||||||
type NextChallengeInput struct {
|
type NextChallengeInput struct {
|
||||||
@@ -49,8 +51,49 @@ func (StubRunner) DiagnoseJobSeeker(context.Context, DiagnosticInput) (Diagnosti
|
|||||||
return DiagnosticResult{}, ErrNotImplemented
|
return DiagnosticResult{}, ErrNotImplemented
|
||||||
}
|
}
|
||||||
|
|
||||||
func (StubRunner) GradeInterviewAnswer(context.Context, GradeAnswerInput) (GradedAnswer, error) {
|
func (StubRunner) GradeInterviewAnswer(_ context.Context, input GradeAnswerInput) (GradedAnswer, error) {
|
||||||
return GradedAnswer{}, ErrNotImplemented
|
wordCount := len(strings.Fields(input.AnswerText))
|
||||||
|
overall := AnswerPartial
|
||||||
|
if wordCount >= 18 {
|
||||||
|
overall = AnswerSolid
|
||||||
|
}
|
||||||
|
if wordCount < 8 {
|
||||||
|
overall = AnswerMiss
|
||||||
|
}
|
||||||
|
|
||||||
|
grade := GradedAnswer{
|
||||||
|
AnswerID: input.AnswerID,
|
||||||
|
QuestionID: input.QuestionID,
|
||||||
|
Concepts: append([]ConceptRef(nil), input.Concepts...),
|
||||||
|
Scores: AnswerScores{
|
||||||
|
Correctness: scoreFromWords(wordCount, 8),
|
||||||
|
Depth: scoreFromWords(wordCount, 14),
|
||||||
|
Communication: scoreFromWords(wordCount, 10),
|
||||||
|
ProductionJudgment: scoreFromWords(wordCount, 20),
|
||||||
|
},
|
||||||
|
Overall: overall,
|
||||||
|
Strengths: []string{"Answer was captured and evaluated through the typed workflow boundary."},
|
||||||
|
Gaps: []string{},
|
||||||
|
Evidence: []EvidenceRef{
|
||||||
|
{
|
||||||
|
Kind: EvidenceAnswer,
|
||||||
|
ID: input.AnswerID,
|
||||||
|
Quote: input.AnswerText,
|
||||||
|
Confidence: 1,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
FollowUp: FollowUpRecommendation{},
|
||||||
|
}
|
||||||
|
|
||||||
|
if overall == AnswerMiss || overall == AnswerPartial {
|
||||||
|
grade.Gaps = []string{"Answer needs more concrete reasoning and tradeoff discussion."}
|
||||||
|
grade.FollowUp = FollowUpRecommendation{
|
||||||
|
Needed: true,
|
||||||
|
Question: "Can you give a concrete production example and explain the tradeoff?",
|
||||||
|
Purpose: FollowUpRepair,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return grade, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (StubRunner) ExtractLearningMemory(context.Context, GradedAnswer) (MemoryUpdateCandidate, error) {
|
func (StubRunner) ExtractLearningMemory(context.Context, GradedAnswer) (MemoryUpdateCandidate, error) {
|
||||||
@@ -64,3 +107,13 @@ func (StubRunner) SelectNextChallenge(context.Context, NextChallengeInput) (Next
|
|||||||
func (StubRunner) UpdateReadinessMap(context.Context, ReadinessUpdateInput) (ReadinessUpdate, error) {
|
func (StubRunner) UpdateReadinessMap(context.Context, ReadinessUpdateInput) (ReadinessUpdate, error) {
|
||||||
return ReadinessUpdate{}, ErrNotImplemented
|
return ReadinessUpdate{}, ErrNotImplemented
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func scoreFromWords(wordCount int, target int) int {
|
||||||
|
if wordCount >= target {
|
||||||
|
return 4
|
||||||
|
}
|
||||||
|
if wordCount >= target/2 {
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestStubRunnerReturnsTypedNotImplemented(t *testing.T) {
|
func TestStubRunnerDiagnoseReturnsTypedNotImplemented(t *testing.T) {
|
||||||
runner := NewStubRunner()
|
runner := NewStubRunner()
|
||||||
|
|
||||||
_, err := runner.DiagnoseJobSeeker(context.Background(), DiagnosticInput{
|
_, err := runner.DiagnoseJobSeeker(context.Background(), DiagnosticInput{
|
||||||
@@ -19,3 +19,28 @@ func TestStubRunnerReturnsTypedNotImplemented(t *testing.T) {
|
|||||||
t.Fatalf("err = %v, want %v", err, ErrNotImplemented)
|
t.Fatalf("err = %v, want %v", err, ErrNotImplemented)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestStubRunnerGradesAnswer(t *testing.T) {
|
||||||
|
runner := NewStubRunner()
|
||||||
|
|
||||||
|
grade, err := runner.GradeInterviewAnswer(context.Background(), GradeAnswerInput{
|
||||||
|
QuestionID: "q-1",
|
||||||
|
AnswerID: "a-1",
|
||||||
|
AnswerText: "Indexes can speed reads by helping the database find rows, but they add write overhead.",
|
||||||
|
Concepts: []ConceptRef{
|
||||||
|
{ID: "database-indexes", Label: "Database indexes", Track: "backend-developer"},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GradeInterviewAnswer error: %v", err)
|
||||||
|
}
|
||||||
|
if grade.AnswerID != "a-1" {
|
||||||
|
t.Fatalf("AnswerID = %q", grade.AnswerID)
|
||||||
|
}
|
||||||
|
if len(grade.Concepts) != 1 {
|
||||||
|
t.Fatalf("concepts = %d, want 1", len(grade.Concepts))
|
||||||
|
}
|
||||||
|
if len(grade.Evidence) != 1 {
|
||||||
|
t.Fatalf("evidence = %d, want 1", len(grade.Evidence))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user