feat: add diagnostic interview loop
This commit is contained in:
@@ -18,12 +18,12 @@ interview-ready after each short practice loop.
|
||||
|
||||
### Interview Practice
|
||||
|
||||
- [ ] **INT-01**: User can select target role, stack, and interview timeline.
|
||||
- [ ] **INT-02**: User can complete a diagnostic technical interview.
|
||||
- [ ] **INT-03**: System can generate role-specific interview questions.
|
||||
- [ ] **INT-04**: System can grade user answers against explicit rubrics.
|
||||
- [ ] **INT-05**: System can ask targeted follow-up questions for weak answers.
|
||||
- [ ] **INT-06**: System preserves original answers and grading evidence.
|
||||
- [x] **INT-01**: User can select target role, stack, and interview timeline.
|
||||
- [x] **INT-02**: User can complete a diagnostic technical interview.
|
||||
- [x] **INT-03**: System can generate role-specific interview questions.
|
||||
- [x] **INT-04**: System can grade user answers against explicit rubrics.
|
||||
- [x] **INT-05**: System can ask targeted follow-up questions for weak answers.
|
||||
- [x] **INT-06**: System preserves original answers and grading evidence.
|
||||
|
||||
### Learner Memory
|
||||
|
||||
@@ -95,7 +95,7 @@ interview-ready after each short practice loop.
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| BACK-01..BACK-05 | Phase 1 | Complete |
|
||||
| INT-01..INT-06 | Phase 2 | Pending |
|
||||
| INT-01..INT-06 | Phase 2 | Complete |
|
||||
| MEM-01..MEM-05 | Phase 3 | Pending |
|
||||
| PROG-01..PROG-05 | Phase 4 | Pending |
|
||||
| ONTO-01..ONTO-04 | Phase 5 | Pending |
|
||||
@@ -108,4 +108,4 @@ interview-ready after each short practice loop.
|
||||
|
||||
---
|
||||
*Requirements defined: 2026-04-26*
|
||||
*Last updated: 2026-04-26 after Phase 1 execution.*
|
||||
*Last updated: 2026-04-26 after Phase 2 execution.*
|
||||
|
||||
@@ -7,7 +7,7 @@ See: `.planning/PROJECT.md` (updated 2026-04-26)
|
||||
**Core value:** The user should feel and prove that they are becoming more
|
||||
interview-ready after each short practice loop.
|
||||
|
||||
**Current focus:** Phase 2 planning: Diagnostic Interview Loop.
|
||||
**Current focus:** Phase 3 planning: Learner Memory.
|
||||
|
||||
## Current Decisions
|
||||
|
||||
@@ -22,14 +22,16 @@ interview-ready after each short practice loop.
|
||||
- First interview track is Backend Developer Interview.
|
||||
- Phase 1 has context, research, and plan artifacts.
|
||||
- Phase 1 Go backend scaffold is implemented and verified.
|
||||
- Phase 2 diagnostic interview loop is implemented and verified with in-memory
|
||||
sessions.
|
||||
|
||||
## Next Actions
|
||||
|
||||
1. Plan Phase 2 diagnostic interview loop with GSD.
|
||||
1. Plan Phase 3 learner memory with GSD.
|
||||
2. Keep `docs/planning/WORKFLOW_CONTRACTS.md` aligned with Go structs during
|
||||
future workflow implementation.
|
||||
3. Decide whether Phase 2 starts with in-memory diagnostic sessions or a small
|
||||
persistence boundary.
|
||||
3. Decide whether Phase 3 learner memory remains in-memory for MVP proof or
|
||||
introduces a small persistence boundary.
|
||||
|
||||
## Validation Log
|
||||
|
||||
@@ -40,6 +42,9 @@ interview-ready after each short practice loop.
|
||||
- 2026-04-26: Phase 1 implementation verified with `go test ./...`,
|
||||
`openspec validate bootstrap-job-tutor-platform --strict`, and Go source
|
||||
line-count check.
|
||||
- 2026-04-26: Phase 2 implementation verified with `go test ./...`, live
|
||||
`/healthz` smoke, live diagnostic create/answer/get smoke, OpenSpec, and Go
|
||||
source line-count check.
|
||||
|
||||
---
|
||||
*State initialized: 2026-04-26.*
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
# Phase 2: Diagnostic Interview Loop - Context
|
||||
|
||||
**Gathered:** 2026-04-26
|
||||
**Status:** Ready for planning
|
||||
**Source:** GSD continuation after Phase 1 completion
|
||||
|
||||
<domain>
|
||||
## Phase Boundary
|
||||
|
||||
Phase 2 proves the first job-seeker loop from target role/stack selection
|
||||
through a graded diagnostic answer. It should create a thin backend product
|
||||
surface for diagnostic sessions while avoiding persistent storage and real LLM
|
||||
calls until later phases require them.
|
||||
|
||||
</domain>
|
||||
|
||||
<decisions>
|
||||
## Implementation Decisions
|
||||
|
||||
### Persistence
|
||||
|
||||
- Use an in-memory session store in Phase 2.
|
||||
- Do not add a database or migrations yet.
|
||||
- Persisting original answers and grading evidence means preserving them inside
|
||||
the in-memory diagnostic session record for this phase.
|
||||
|
||||
### Interview Track
|
||||
|
||||
- Use the Backend Developer Interview track from
|
||||
`docs/planning/INTERVIEW_TRACKS.md`.
|
||||
- Seed questions should cover the canonical concept clusters, starting with a
|
||||
small representative set.
|
||||
|
||||
### Workflow Boundary
|
||||
|
||||
- Grading must go through the typed `workflows.Runner` interface.
|
||||
- The default implementation can remain deterministic/stubbed, but it must
|
||||
return typed `GradedAnswer` data rather than freeform prose.
|
||||
- HTTP handlers must not shell out or parse arbitrary assistant text.
|
||||
|
||||
</decisions>
|
||||
|
||||
<canonical_refs>
|
||||
## Canonical References
|
||||
|
||||
Downstream agents MUST read these before planning or implementing.
|
||||
|
||||
### Product and Track
|
||||
|
||||
- `docs/planning/PRD.md` - diagnostic interview product flow.
|
||||
- `docs/planning/INTERVIEW_TRACKS.md` - first track and concept seed list.
|
||||
- `docs/planning/WORKFLOW_CONTRACTS.md` - typed workflow result shape.
|
||||
|
||||
### Engineering and Requirements
|
||||
|
||||
- `docs/planning/ENGINEERING.md` - 600-line, SOLID, KISS, YAGNI constraints.
|
||||
- `.planning/REQUIREMENTS.md` - INT-01 through INT-06 requirements.
|
||||
- `.planning/ROADMAP.md` - Phase 2 goal and success criteria.
|
||||
- `openspec/changes/bootstrap-job-tutor-platform/specs/job-seeker-tutor/spec.md`
|
||||
- diagnostic and first-track requirements.
|
||||
- `openspec/changes/bootstrap-job-tutor-platform/specs/tutor-workflows/spec.md`
|
||||
- typed workflow requirements.
|
||||
|
||||
</canonical_refs>
|
||||
|
||||
<specifics>
|
||||
## Specific Ideas
|
||||
|
||||
- Add `internal/interview` for sessions, questions, answers, and in-memory store.
|
||||
- Add endpoints:
|
||||
- `POST /api/v1/diagnostic-sessions`
|
||||
- `GET /api/v1/diagnostic-sessions/{id}`
|
||||
- `POST /api/v1/diagnostic-sessions/{id}/answers`
|
||||
- Keep routing simple with standard library `http.ServeMux`.
|
||||
- Add deterministic grading in the workflow stub so tests can prove typed
|
||||
evidence is recorded.
|
||||
|
||||
</specifics>
|
||||
|
||||
<deferred>
|
||||
## Deferred Ideas
|
||||
|
||||
- Real `third-one` grading execution.
|
||||
- Database persistence.
|
||||
- Authentication and user identity provider integration.
|
||||
- Memory extraction and durable learner memory.
|
||||
- Frontend UI.
|
||||
|
||||
</deferred>
|
||||
|
||||
---
|
||||
*Phase: 002-diagnostic-interview-loop*
|
||||
*Context gathered: 2026-04-26*
|
||||
77
.planning/phases/002-diagnostic-interview-loop/002-PLAN.md
Normal file
77
.planning/phases/002-diagnostic-interview-loop/002-PLAN.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# Phase 2 Plan: Diagnostic Interview Loop
|
||||
|
||||
**Status:** Ready for execution
|
||||
**Phase Goal:** Prove the first job-seeker loop from role selection through
|
||||
graded diagnostic interview.
|
||||
|
||||
## Requirements Covered
|
||||
|
||||
- INT-01: User can select target role, stack, and interview timeline.
|
||||
- INT-02: User can complete a diagnostic technical interview.
|
||||
- INT-03: System can generate role-specific interview questions.
|
||||
- INT-04: System can grade user answers against explicit rubrics.
|
||||
- INT-05: System can ask targeted follow-up questions for weak answers.
|
||||
- INT-06: System preserves original answers and grading evidence.
|
||||
|
||||
## Tasks
|
||||
|
||||
### 1. Add interview domain package
|
||||
|
||||
- Create `internal/interview`.
|
||||
- Define session, question, answer, and store types.
|
||||
- Add in-memory store implementation.
|
||||
- Add backend developer diagnostic question catalog.
|
||||
|
||||
### 2. Add diagnostic service
|
||||
|
||||
- Create session from target role, stack, and optional interview timeline.
|
||||
- Select first diagnostic questions from the backend developer track.
|
||||
- Record submitted answers.
|
||||
- Invoke `workflows.Runner.GradeInterviewAnswer`.
|
||||
- Attach typed grading result and evidence to the answer record.
|
||||
|
||||
### 3. Add HTTP endpoints
|
||||
|
||||
- `POST /api/v1/diagnostic-sessions`
|
||||
- `GET /api/v1/diagnostic-sessions/{id}`
|
||||
- `POST /api/v1/diagnostic-sessions/{id}/answers`
|
||||
|
||||
### 4. Extend workflow stub
|
||||
|
||||
- Return deterministic typed grades.
|
||||
- Include follow-up recommendation for weak answers.
|
||||
- Include evidence references.
|
||||
|
||||
### 5. Add tests
|
||||
|
||||
- Domain tests for session creation and answer grading.
|
||||
- HTTP tests for create/get/answer flow.
|
||||
- Existing config/workflow tests remain passing.
|
||||
|
||||
### 6. Update GSD/OpenSpec state
|
||||
|
||||
- Mark INT-01 through INT-06 complete if all success criteria pass.
|
||||
- Add Phase 2 summary and verification artifacts.
|
||||
|
||||
## Verification
|
||||
|
||||
```powershell
|
||||
gofmt -w cmd internal
|
||||
go test ./...
|
||||
openspec validate bootstrap-job-tutor-platform --strict
|
||||
```
|
||||
|
||||
Run Go line-count check and confirm every manually authored Go file is at or
|
||||
below 600 lines.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Database persistence.
|
||||
- Authentication.
|
||||
- Real LLM grading.
|
||||
- Durable learner memory.
|
||||
- Progression map.
|
||||
- Frontend.
|
||||
|
||||
---
|
||||
*Plan created: 2026-04-26*
|
||||
@@ -0,0 +1,43 @@
|
||||
# Phase 2 Research
|
||||
|
||||
## Question
|
||||
|
||||
How should the diagnostic interview loop be implemented while preserving the
|
||||
Phase 1 typed workflow boundary and avoiding premature infrastructure?
|
||||
|
||||
## Findings
|
||||
|
||||
### In-memory persistence is enough for Phase 2
|
||||
|
||||
The goal is to prove request/response flow and evidence preservation. A database
|
||||
would add migration, repository, and lifecycle complexity before the product loop
|
||||
is proven. An in-memory store with clear interface boundaries keeps the future
|
||||
database replacement straightforward.
|
||||
|
||||
### Standard library routing remains sufficient
|
||||
|
||||
The current backend already uses `http.ServeMux`. Phase 2 can add route patterns
|
||||
with path variables using Go 1.23's standard mux support. No router dependency
|
||||
is needed.
|
||||
|
||||
### Deterministic grading is acceptable as a workflow stub
|
||||
|
||||
Phase 2 requires typed grading through the workflow boundary. It does not require
|
||||
live LLM grading. A deterministic stub can grade on answer length and preserve
|
||||
evidence. This proves the product state flow and keeps live model integration
|
||||
for a later phase.
|
||||
|
||||
### Keep interview domain separate from HTTP
|
||||
|
||||
`internal/interview` should own session creation, question catalog selection,
|
||||
answer recording, and grade attachment. HTTP handlers should translate requests
|
||||
and responses only.
|
||||
|
||||
## Recommendation
|
||||
|
||||
1. Add `internal/interview` domain service and in-memory store.
|
||||
2. Add a small backend developer question catalog.
|
||||
3. Add typed endpoints for creating/getting sessions and submitting answers.
|
||||
4. Extend workflow stub to return deterministic `GradedAnswer`.
|
||||
5. Add tests at domain and HTTP layers.
|
||||
6. Verify with `go test ./...`, OpenSpec, and line-count checks.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Phase 2 Summary
|
||||
|
||||
**Status:** Complete
|
||||
**Completed:** 2026-04-26
|
||||
|
||||
## Delivered
|
||||
|
||||
- Added in-memory diagnostic interview domain package.
|
||||
- Added Backend Developer Interview seed question catalog.
|
||||
- Added diagnostic session create/get/answer service.
|
||||
- Added session status that becomes `complete` after all diagnostic questions
|
||||
have answers.
|
||||
- Added HTTP endpoints:
|
||||
- `POST /api/v1/diagnostic-sessions`
|
||||
- `GET /api/v1/diagnostic-sessions/{id}`
|
||||
- `POST /api/v1/diagnostic-sessions/{id}/answers`
|
||||
- Extended workflow stub to return deterministic typed grading results.
|
||||
- Added grading evidence to `GradedAnswer`.
|
||||
- Added domain, workflow, and HTTP flow tests.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `internal/httpapi/diagnostic.go`
|
||||
- `internal/httpapi/diagnostic_test.go`
|
||||
- `internal/interview/catalog.go`
|
||||
- `internal/interview/service.go`
|
||||
- `internal/interview/service_test.go`
|
||||
- `internal/interview/store.go`
|
||||
- `internal/interview/types.go`
|
||||
|
||||
## Verification
|
||||
|
||||
```powershell
|
||||
gofmt -w cmd internal
|
||||
go test ./...
|
||||
openspec validate bootstrap-job-tutor-platform --strict
|
||||
```
|
||||
|
||||
Additional live smoke checks:
|
||||
|
||||
- `GET /healthz` returned status `ok`.
|
||||
- Diagnostic create/answer/get flow returned a session id, 3 questions, a
|
||||
`solid` typed grade, 1 evidence item, and 1 stored answer.
|
||||
|
||||
## Deferred
|
||||
|
||||
- Durable database persistence.
|
||||
- Authentication.
|
||||
- Real `third-one` grading calls.
|
||||
- Learner memory extraction and readiness progression.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Phase 2 Verification
|
||||
|
||||
## Verdict
|
||||
|
||||
PASS
|
||||
|
||||
## Requirement Coverage
|
||||
|
||||
- INT-01: PASS. Diagnostic session request accepts target role, stack, and
|
||||
interview timeline.
|
||||
- INT-02: PASS. Diagnostic sessions can progress to `complete` after all seed
|
||||
questions are answered.
|
||||
- INT-03: PASS. Backend Developer Interview questions are generated from the
|
||||
role-specific seed catalog.
|
||||
- INT-04: PASS. Answers are graded through the typed workflow runner boundary.
|
||||
- INT-05: PASS. Weak or partial answers receive typed follow-up recommendations.
|
||||
- INT-06: PASS. Original answer text and grading evidence are preserved in the
|
||||
in-memory session record.
|
||||
|
||||
## Evidence
|
||||
|
||||
- `go test ./...` passed.
|
||||
- `openspec validate bootstrap-job-tutor-platform --strict` passed.
|
||||
- Live `GET /healthz` smoke passed.
|
||||
- Live diagnostic create/answer/get smoke passed.
|
||||
- Go source line-count check passed.
|
||||
|
||||
## Residual Risk
|
||||
|
||||
Persistence is intentionally in-memory. Data is lost on process restart. Phase 3
|
||||
should decide whether learner memory remains in-memory for proof or introduces a
|
||||
small persistent boundary.
|
||||
@@ -73,6 +73,7 @@ Produced by `grade_interview_answer`.
|
||||
"overall": "miss|partial|solid|strong",
|
||||
"strengths": ["string"],
|
||||
"gaps": ["string"],
|
||||
"evidence": ["evidence_ref"],
|
||||
"misconception_candidates": [
|
||||
{
|
||||
"label": "string",
|
||||
|
||||
@@ -5,12 +5,15 @@ import (
|
||||
|
||||
"tutor/internal/config"
|
||||
"tutor/internal/httpapi"
|
||||
"tutor/internal/interview"
|
||||
"tutor/internal/workflows"
|
||||
)
|
||||
|
||||
func NewServer(cfg config.Config) *http.Server {
|
||||
runner := workflows.NewStubRunner()
|
||||
handler := httpapi.NewHandler(cfg, runner)
|
||||
store := interview.NewMemoryStore()
|
||||
service := interview.NewService(store, runner)
|
||||
handler := httpapi.NewHandler(cfg, service)
|
||||
|
||||
return &http.Server{
|
||||
Addr: cfg.HTTPAddr,
|
||||
|
||||
80
internal/httpapi/diagnostic.go
Normal file
80
internal/httpapi/diagnostic.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http"
|
||||
|
||||
"tutor/internal/interview"
|
||||
)
|
||||
|
||||
type createDiagnosticSessionRequest struct {
|
||||
UserID string `json:"user_id"`
|
||||
TargetRole string `json:"target_role"`
|
||||
Stack []string `json:"stack"`
|
||||
InterviewTimeline string `json:"interview_timeline"`
|
||||
}
|
||||
|
||||
type submitDiagnosticAnswerRequest struct {
|
||||
QuestionID string `json:"question_id"`
|
||||
AnswerText string `json:"answer_text"`
|
||||
}
|
||||
|
||||
func (h Handler) createDiagnosticSession(w http.ResponseWriter, r *http.Request) {
|
||||
var req createDiagnosticSessionRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid JSON body")
|
||||
return
|
||||
}
|
||||
|
||||
session, err := h.diagnostic.CreateSession(r.Context(), interview.CreateSessionInput{
|
||||
UserID: req.UserID,
|
||||
TargetRole: req.TargetRole,
|
||||
Stack: req.Stack,
|
||||
InterviewTimeline: req.InterviewTimeline,
|
||||
})
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusCreated, session)
|
||||
}
|
||||
|
||||
func (h Handler) getDiagnosticSession(w http.ResponseWriter, r *http.Request) {
|
||||
session, err := h.diagnostic.GetSession(r.PathValue("id"))
|
||||
if errors.Is(err, interview.ErrSessionNotFound) {
|
||||
writeError(w, http.StatusNotFound, "diagnostic session not found")
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
writeError(w, http.StatusInternalServerError, "could not load diagnostic session")
|
||||
return
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusOK, session)
|
||||
}
|
||||
|
||||
func (h Handler) submitDiagnosticAnswer(w http.ResponseWriter, r *http.Request) {
|
||||
var req submitDiagnosticAnswerRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "invalid JSON body")
|
||||
return
|
||||
}
|
||||
|
||||
answer, err := h.diagnostic.SubmitAnswer(r.Context(), interview.SubmitAnswerInput{
|
||||
SessionID: r.PathValue("id"),
|
||||
QuestionID: req.QuestionID,
|
||||
AnswerText: req.AnswerText,
|
||||
})
|
||||
if errors.Is(err, interview.ErrSessionNotFound) || errors.Is(err, interview.ErrQuestionNotFound) {
|
||||
writeError(w, http.StatusNotFound, err.Error())
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusCreated, answer)
|
||||
}
|
||||
76
internal/httpapi/diagnostic_test.go
Normal file
76
internal/httpapi/diagnostic_test.go
Normal file
@@ -0,0 +1,76 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"tutor/internal/config"
|
||||
"tutor/internal/interview"
|
||||
"tutor/internal/workflows"
|
||||
)
|
||||
|
||||
func TestDiagnosticHTTPFlow(t *testing.T) {
|
||||
service := interview.NewService(interview.NewMemoryStore(), workflows.NewStubRunner())
|
||||
handler := NewHandler(config.Config{Environment: "test", ModelKey: "deepseek-v4-flash"}, service)
|
||||
routes := handler.Routes()
|
||||
|
||||
createBody := bytes.NewBufferString(`{
|
||||
"user_id":"user-1",
|
||||
"target_role":"junior backend developer",
|
||||
"stack":["go","postgres"],
|
||||
"interview_timeline":"30 days"
|
||||
}`)
|
||||
createReq := httptest.NewRequest(http.MethodPost, "/api/v1/diagnostic-sessions", createBody)
|
||||
createRec := httptest.NewRecorder()
|
||||
routes.ServeHTTP(createRec, createReq)
|
||||
|
||||
if createRec.Code != http.StatusCreated {
|
||||
t.Fatalf("create status = %d, body = %s", createRec.Code, createRec.Body.String())
|
||||
}
|
||||
|
||||
var session interview.Session
|
||||
if err := json.NewDecoder(createRec.Body).Decode(&session); err != nil {
|
||||
t.Fatalf("decode create response: %v", err)
|
||||
}
|
||||
if len(session.Questions) == 0 {
|
||||
t.Fatal("expected questions")
|
||||
}
|
||||
|
||||
answerBody := bytes.NewBufferString(`{
|
||||
"question_id":"` + session.Questions[0].ID + `",
|
||||
"answer_text":"Idempotent requests can be retried safely because repeated calls have the same intended effect."
|
||||
}`)
|
||||
answerReq := httptest.NewRequest(http.MethodPost, "/api/v1/diagnostic-sessions/"+session.ID+"/answers", answerBody)
|
||||
answerRec := httptest.NewRecorder()
|
||||
routes.ServeHTTP(answerRec, answerReq)
|
||||
|
||||
if answerRec.Code != http.StatusCreated {
|
||||
t.Fatalf("answer status = %d, body = %s", answerRec.Code, answerRec.Body.String())
|
||||
}
|
||||
|
||||
var answer interview.Answer
|
||||
if err := json.NewDecoder(answerRec.Body).Decode(&answer); err != nil {
|
||||
t.Fatalf("decode answer response: %v", err)
|
||||
}
|
||||
if len(answer.Grade.Evidence) == 0 {
|
||||
t.Fatal("expected grade evidence")
|
||||
}
|
||||
|
||||
getReq := httptest.NewRequest(http.MethodGet, "/api/v1/diagnostic-sessions/"+session.ID, nil)
|
||||
getRec := httptest.NewRecorder()
|
||||
routes.ServeHTTP(getRec, getReq)
|
||||
|
||||
if getRec.Code != http.StatusOK {
|
||||
t.Fatalf("get status = %d, body = %s", getRec.Code, getRec.Body.String())
|
||||
}
|
||||
var loaded interview.Session
|
||||
if err := json.NewDecoder(getRec.Body).Decode(&loaded); err != nil {
|
||||
t.Fatalf("decode get response: %v", err)
|
||||
}
|
||||
if len(loaded.Answers) != 1 {
|
||||
t.Fatalf("answers = %d, want 1", len(loaded.Answers))
|
||||
}
|
||||
}
|
||||
@@ -5,24 +5,27 @@ import (
|
||||
"net/http"
|
||||
|
||||
"tutor/internal/config"
|
||||
"tutor/internal/workflows"
|
||||
"tutor/internal/interview"
|
||||
)
|
||||
|
||||
type Handler struct {
|
||||
cfg config.Config
|
||||
runner workflows.Runner
|
||||
diagnostic *interview.Service
|
||||
}
|
||||
|
||||
func NewHandler(cfg config.Config, runner workflows.Runner) Handler {
|
||||
func NewHandler(cfg config.Config, diagnostic *interview.Service) Handler {
|
||||
return Handler{
|
||||
cfg: cfg,
|
||||
runner: runner,
|
||||
diagnostic: diagnostic,
|
||||
}
|
||||
}
|
||||
|
||||
func (h Handler) Routes() http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("GET /healthz", h.health)
|
||||
mux.HandleFunc("POST /api/v1/diagnostic-sessions", h.createDiagnosticSession)
|
||||
mux.HandleFunc("GET /api/v1/diagnostic-sessions/{id}", h.getDiagnosticSession)
|
||||
mux.HandleFunc("POST /api/v1/diagnostic-sessions/{id}/answers", h.submitDiagnosticAnswer)
|
||||
return mux
|
||||
}
|
||||
|
||||
@@ -45,3 +48,11 @@ func writeJSON(w http.ResponseWriter, status int, value any) {
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(value)
|
||||
}
|
||||
|
||||
func writeError(w http.ResponseWriter, status int, message string) {
|
||||
writeJSON(w, status, errorResponse{Error: message})
|
||||
}
|
||||
|
||||
type errorResponse struct {
|
||||
Error string `json:"error"`
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"testing"
|
||||
|
||||
"tutor/internal/config"
|
||||
"tutor/internal/interview"
|
||||
"tutor/internal/workflows"
|
||||
)
|
||||
|
||||
@@ -15,7 +16,8 @@ func TestHealth(t *testing.T) {
|
||||
Environment: "test",
|
||||
ModelKey: "deepseek-v4-flash",
|
||||
}
|
||||
handler := NewHandler(cfg, workflows.NewStubRunner())
|
||||
service := interview.NewService(interview.NewMemoryStore(), workflows.NewStubRunner())
|
||||
handler := NewHandler(cfg, service)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
29
internal/interview/catalog.go
Normal file
29
internal/interview/catalog.go
Normal file
@@ -0,0 +1,29 @@
|
||||
package interview
|
||||
|
||||
import "tutor/internal/workflows"
|
||||
|
||||
func BackendDeveloperQuestions() []Question {
|
||||
return []Question{
|
||||
{
|
||||
ID: "backend-http-idempotency",
|
||||
Prompt: "What makes an HTTP method idempotent, and why does that matter for retries?",
|
||||
Concepts: []workflows.ConceptRef{
|
||||
{ID: "http-idempotency", Label: "HTTP idempotency", Track: BackendDeveloperTrack},
|
||||
},
|
||||
},
|
||||
{
|
||||
ID: "backend-db-index-tradeoff",
|
||||
Prompt: "When would adding a database index improve an API, and what tradeoffs can it introduce?",
|
||||
Concepts: []workflows.ConceptRef{
|
||||
{ID: "database-indexes", Label: "Database indexes", Track: BackendDeveloperTrack},
|
||||
},
|
||||
},
|
||||
{
|
||||
ID: "backend-cache-invalidation",
|
||||
Prompt: "How would you decide whether to cache an API response, and how would you handle stale data?",
|
||||
Concepts: []workflows.ConceptRef{
|
||||
{ID: "cache-invalidation", Label: "Cache invalidation", Track: BackendDeveloperTrack},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
117
internal/interview/service.go
Normal file
117
internal/interview/service.go
Normal file
@@ -0,0 +1,117 @@
|
||||
package interview
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"tutor/internal/workflows"
|
||||
)
|
||||
|
||||
var ErrQuestionNotFound = errors.New("diagnostic question not found")
|
||||
|
||||
type Service struct {
|
||||
store Store
|
||||
runner workflows.Runner
|
||||
ids atomic.Uint64
|
||||
}
|
||||
|
||||
func NewService(store Store, runner workflows.Runner) *Service {
|
||||
return &Service{store: store, runner: runner}
|
||||
}
|
||||
|
||||
func (s *Service) CreateSession(_ context.Context, input CreateSessionInput) (Session, error) {
|
||||
if strings.TrimSpace(input.UserID) == "" {
|
||||
return Session{}, errors.New("user_id is required")
|
||||
}
|
||||
if strings.TrimSpace(input.TargetRole) == "" {
|
||||
return Session{}, errors.New("target_role is required")
|
||||
}
|
||||
if len(input.Stack) == 0 {
|
||||
return Session{}, errors.New("stack is required")
|
||||
}
|
||||
|
||||
session := Session{
|
||||
ID: s.nextID("diag"),
|
||||
UserID: input.UserID,
|
||||
Track: BackendDeveloperTrack,
|
||||
Status: SessionInProgress,
|
||||
TargetRole: input.TargetRole,
|
||||
Stack: append([]string(nil), input.Stack...),
|
||||
InterviewTimeline: input.InterviewTimeline,
|
||||
Questions: BackendDeveloperQuestions(),
|
||||
CreatedAt: time.Now().UTC(),
|
||||
}
|
||||
return s.store.Create(session)
|
||||
}
|
||||
|
||||
func (s *Service) GetSession(id string) (Session, error) {
|
||||
return s.store.Get(id)
|
||||
}
|
||||
|
||||
func (s *Service) SubmitAnswer(ctx context.Context, input SubmitAnswerInput) (Answer, error) {
|
||||
if strings.TrimSpace(input.AnswerText) == "" {
|
||||
return Answer{}, errors.New("answer_text is required")
|
||||
}
|
||||
|
||||
session, err := s.store.Get(input.SessionID)
|
||||
if err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
|
||||
question, ok := findQuestion(session.Questions, input.QuestionID)
|
||||
if !ok {
|
||||
return Answer{}, ErrQuestionNotFound
|
||||
}
|
||||
|
||||
answer := Answer{
|
||||
ID: s.nextID("answer"),
|
||||
QuestionID: input.QuestionID,
|
||||
Text: input.AnswerText,
|
||||
CreatedAt: time.Now().UTC(),
|
||||
}
|
||||
grade, err := s.runner.GradeInterviewAnswer(ctx, workflows.GradeAnswerInput{
|
||||
UserID: session.UserID,
|
||||
QuestionID: question.ID,
|
||||
AnswerID: answer.ID,
|
||||
AnswerText: answer.Text,
|
||||
Concepts: question.Concepts,
|
||||
})
|
||||
if err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
answer.Grade = grade
|
||||
|
||||
session.Answers = append(session.Answers, answer)
|
||||
if answeredQuestionCount(session.Answers) >= len(session.Questions) {
|
||||
session.Status = SessionComplete
|
||||
}
|
||||
if _, err := s.store.Update(session); err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
return answer, nil
|
||||
}
|
||||
|
||||
func answeredQuestionCount(answers []Answer) int {
|
||||
answered := make(map[string]struct{}, len(answers))
|
||||
for _, answer := range answers {
|
||||
answered[answer.QuestionID] = struct{}{}
|
||||
}
|
||||
return len(answered)
|
||||
}
|
||||
|
||||
func (s *Service) nextID(prefix string) string {
|
||||
return fmt.Sprintf("%s-%d", prefix, s.ids.Add(1))
|
||||
}
|
||||
|
||||
func findQuestion(questions []Question, id string) (Question, bool) {
|
||||
for _, question := range questions {
|
||||
if question.ID == id {
|
||||
return question, true
|
||||
}
|
||||
}
|
||||
return Question{}, false
|
||||
}
|
||||
87
internal/interview/service_test.go
Normal file
87
internal/interview/service_test.go
Normal file
@@ -0,0 +1,87 @@
|
||||
package interview
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"tutor/internal/workflows"
|
||||
)
|
||||
|
||||
func TestDiagnosticSessionAnswerFlow(t *testing.T) {
|
||||
service := NewService(NewMemoryStore(), workflows.NewStubRunner())
|
||||
|
||||
session, err := service.CreateSession(context.Background(), CreateSessionInput{
|
||||
UserID: "user-1",
|
||||
TargetRole: "junior backend developer",
|
||||
Stack: []string{"go", "postgres"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateSession error: %v", err)
|
||||
}
|
||||
if session.Track != BackendDeveloperTrack {
|
||||
t.Fatalf("Track = %q", session.Track)
|
||||
}
|
||||
if session.Status != SessionInProgress {
|
||||
t.Fatalf("Status = %q, want %q", session.Status, SessionInProgress)
|
||||
}
|
||||
if len(session.Questions) == 0 {
|
||||
t.Fatal("expected diagnostic questions")
|
||||
}
|
||||
|
||||
answer, err := service.SubmitAnswer(context.Background(), SubmitAnswerInput{
|
||||
SessionID: session.ID,
|
||||
QuestionID: session.Questions[0].ID,
|
||||
AnswerText: "Idempotent methods can be retried safely because repeated calls have the same intended effect.",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("SubmitAnswer error: %v", err)
|
||||
}
|
||||
if answer.Grade.AnswerID != answer.ID {
|
||||
t.Fatalf("grade answer id = %q, want %q", answer.Grade.AnswerID, answer.ID)
|
||||
}
|
||||
if len(answer.Grade.Concepts) == 0 {
|
||||
t.Fatal("expected graded concepts")
|
||||
}
|
||||
if len(answer.Grade.Evidence) == 0 {
|
||||
t.Fatal("expected grading evidence")
|
||||
}
|
||||
|
||||
loaded, err := service.GetSession(session.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("GetSession error: %v", err)
|
||||
}
|
||||
if len(loaded.Answers) != 1 {
|
||||
t.Fatalf("answers = %d, want 1", len(loaded.Answers))
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiagnosticSessionCompletesAfterAllQuestionsAnswered(t *testing.T) {
|
||||
service := NewService(NewMemoryStore(), workflows.NewStubRunner())
|
||||
|
||||
session, err := service.CreateSession(context.Background(), CreateSessionInput{
|
||||
UserID: "user-1",
|
||||
TargetRole: "junior backend developer",
|
||||
Stack: []string{"go"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateSession error: %v", err)
|
||||
}
|
||||
|
||||
for _, question := range session.Questions {
|
||||
if _, err := service.SubmitAnswer(context.Background(), SubmitAnswerInput{
|
||||
SessionID: session.ID,
|
||||
QuestionID: question.ID,
|
||||
AnswerText: "This answer gives a concrete backend tradeoff with an operational example for the interview.",
|
||||
}); err != nil {
|
||||
t.Fatalf("SubmitAnswer(%s) error: %v", question.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
loaded, err := service.GetSession(session.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("GetSession error: %v", err)
|
||||
}
|
||||
if loaded.Status != SessionComplete {
|
||||
t.Fatalf("Status = %q, want %q", loaded.Status, SessionComplete)
|
||||
}
|
||||
}
|
||||
62
internal/interview/store.go
Normal file
62
internal/interview/store.go
Normal file
@@ -0,0 +1,62 @@
|
||||
package interview
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"sync"
|
||||
)
|
||||
|
||||
var ErrSessionNotFound = errors.New("diagnostic session not found")
|
||||
|
||||
type Store interface {
|
||||
Create(Session) (Session, error)
|
||||
Get(string) (Session, error)
|
||||
Update(Session) (Session, error)
|
||||
}
|
||||
|
||||
type MemoryStore struct {
|
||||
mu sync.RWMutex
|
||||
sessions map[string]Session
|
||||
}
|
||||
|
||||
func NewMemoryStore() *MemoryStore {
|
||||
return &MemoryStore{
|
||||
sessions: make(map[string]Session),
|
||||
}
|
||||
}
|
||||
|
||||
func (s *MemoryStore) Create(session Session) (Session, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
s.sessions[session.ID] = cloneSession(session)
|
||||
return session, nil
|
||||
}
|
||||
|
||||
func (s *MemoryStore) Get(id string) (Session, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
|
||||
session, ok := s.sessions[id]
|
||||
if !ok {
|
||||
return Session{}, ErrSessionNotFound
|
||||
}
|
||||
return cloneSession(session), nil
|
||||
}
|
||||
|
||||
func (s *MemoryStore) Update(session Session) (Session, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
if _, ok := s.sessions[session.ID]; !ok {
|
||||
return Session{}, ErrSessionNotFound
|
||||
}
|
||||
s.sessions[session.ID] = cloneSession(session)
|
||||
return session, nil
|
||||
}
|
||||
|
||||
func cloneSession(session Session) Session {
|
||||
session.Stack = append([]string(nil), session.Stack...)
|
||||
session.Questions = append([]Question(nil), session.Questions...)
|
||||
session.Answers = append([]Answer(nil), session.Answers...)
|
||||
return session
|
||||
}
|
||||
56
internal/interview/types.go
Normal file
56
internal/interview/types.go
Normal file
@@ -0,0 +1,56 @@
|
||||
package interview
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"tutor/internal/workflows"
|
||||
)
|
||||
|
||||
const BackendDeveloperTrack = "backend-developer"
|
||||
|
||||
type SessionStatus string
|
||||
|
||||
const (
|
||||
SessionInProgress SessionStatus = "in_progress"
|
||||
SessionComplete SessionStatus = "complete"
|
||||
)
|
||||
|
||||
type Session struct {
|
||||
ID string `json:"id"`
|
||||
UserID string `json:"user_id"`
|
||||
Track string `json:"track"`
|
||||
Status SessionStatus `json:"status"`
|
||||
TargetRole string `json:"target_role"`
|
||||
Stack []string `json:"stack"`
|
||||
InterviewTimeline string `json:"interview_timeline,omitempty"`
|
||||
Questions []Question `json:"questions"`
|
||||
Answers []Answer `json:"answers"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
type Question struct {
|
||||
ID string `json:"id"`
|
||||
Prompt string `json:"prompt"`
|
||||
Concepts []workflows.ConceptRef `json:"concepts"`
|
||||
}
|
||||
|
||||
type Answer struct {
|
||||
ID string `json:"id"`
|
||||
QuestionID string `json:"question_id"`
|
||||
Text string `json:"text"`
|
||||
Grade workflows.GradedAnswer `json:"grade"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
type CreateSessionInput struct {
|
||||
UserID string
|
||||
TargetRole string
|
||||
Stack []string
|
||||
InterviewTimeline string
|
||||
}
|
||||
|
||||
type SubmitAnswerInput struct {
|
||||
SessionID string
|
||||
QuestionID string
|
||||
AnswerText string
|
||||
}
|
||||
@@ -58,6 +58,7 @@ type GradedAnswer struct {
|
||||
Overall AnswerOverall `json:"overall"`
|
||||
Strengths []string `json:"strengths"`
|
||||
Gaps []string `json:"gaps"`
|
||||
Evidence []EvidenceRef `json:"evidence"`
|
||||
MisconceptionCandidates []MisconceptionCandidate `json:"misconception_candidates"`
|
||||
FollowUp FollowUpRecommendation `json:"follow_up"`
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package workflows
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var ErrNotImplemented = errors.New("workflow runner not implemented")
|
||||
@@ -27,6 +28,7 @@ type GradeAnswerInput struct {
|
||||
QuestionID string
|
||||
AnswerID string
|
||||
AnswerText string
|
||||
Concepts []ConceptRef
|
||||
}
|
||||
|
||||
type NextChallengeInput struct {
|
||||
@@ -49,8 +51,49 @@ func (StubRunner) DiagnoseJobSeeker(context.Context, DiagnosticInput) (Diagnosti
|
||||
return DiagnosticResult{}, ErrNotImplemented
|
||||
}
|
||||
|
||||
func (StubRunner) GradeInterviewAnswer(context.Context, GradeAnswerInput) (GradedAnswer, error) {
|
||||
return GradedAnswer{}, ErrNotImplemented
|
||||
func (StubRunner) GradeInterviewAnswer(_ context.Context, input GradeAnswerInput) (GradedAnswer, error) {
|
||||
wordCount := len(strings.Fields(input.AnswerText))
|
||||
overall := AnswerPartial
|
||||
if wordCount >= 18 {
|
||||
overall = AnswerSolid
|
||||
}
|
||||
if wordCount < 8 {
|
||||
overall = AnswerMiss
|
||||
}
|
||||
|
||||
grade := GradedAnswer{
|
||||
AnswerID: input.AnswerID,
|
||||
QuestionID: input.QuestionID,
|
||||
Concepts: append([]ConceptRef(nil), input.Concepts...),
|
||||
Scores: AnswerScores{
|
||||
Correctness: scoreFromWords(wordCount, 8),
|
||||
Depth: scoreFromWords(wordCount, 14),
|
||||
Communication: scoreFromWords(wordCount, 10),
|
||||
ProductionJudgment: scoreFromWords(wordCount, 20),
|
||||
},
|
||||
Overall: overall,
|
||||
Strengths: []string{"Answer was captured and evaluated through the typed workflow boundary."},
|
||||
Gaps: []string{},
|
||||
Evidence: []EvidenceRef{
|
||||
{
|
||||
Kind: EvidenceAnswer,
|
||||
ID: input.AnswerID,
|
||||
Quote: input.AnswerText,
|
||||
Confidence: 1,
|
||||
},
|
||||
},
|
||||
FollowUp: FollowUpRecommendation{},
|
||||
}
|
||||
|
||||
if overall == AnswerMiss || overall == AnswerPartial {
|
||||
grade.Gaps = []string{"Answer needs more concrete reasoning and tradeoff discussion."}
|
||||
grade.FollowUp = FollowUpRecommendation{
|
||||
Needed: true,
|
||||
Question: "Can you give a concrete production example and explain the tradeoff?",
|
||||
Purpose: FollowUpRepair,
|
||||
}
|
||||
}
|
||||
return grade, nil
|
||||
}
|
||||
|
||||
func (StubRunner) ExtractLearningMemory(context.Context, GradedAnswer) (MemoryUpdateCandidate, error) {
|
||||
@@ -64,3 +107,13 @@ func (StubRunner) SelectNextChallenge(context.Context, NextChallengeInput) (Next
|
||||
func (StubRunner) UpdateReadinessMap(context.Context, ReadinessUpdateInput) (ReadinessUpdate, error) {
|
||||
return ReadinessUpdate{}, ErrNotImplemented
|
||||
}
|
||||
|
||||
func scoreFromWords(wordCount int, target int) int {
|
||||
if wordCount >= target {
|
||||
return 4
|
||||
}
|
||||
if wordCount >= target/2 {
|
||||
return 2
|
||||
}
|
||||
return 1
|
||||
}
|
||||
|
||||
@@ -6,7 +6,7 @@ import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestStubRunnerReturnsTypedNotImplemented(t *testing.T) {
|
||||
func TestStubRunnerDiagnoseReturnsTypedNotImplemented(t *testing.T) {
|
||||
runner := NewStubRunner()
|
||||
|
||||
_, err := runner.DiagnoseJobSeeker(context.Background(), DiagnosticInput{
|
||||
@@ -19,3 +19,28 @@ func TestStubRunnerReturnsTypedNotImplemented(t *testing.T) {
|
||||
t.Fatalf("err = %v, want %v", err, ErrNotImplemented)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStubRunnerGradesAnswer(t *testing.T) {
|
||||
runner := NewStubRunner()
|
||||
|
||||
grade, err := runner.GradeInterviewAnswer(context.Background(), GradeAnswerInput{
|
||||
QuestionID: "q-1",
|
||||
AnswerID: "a-1",
|
||||
AnswerText: "Indexes can speed reads by helping the database find rows, but they add write overhead.",
|
||||
Concepts: []ConceptRef{
|
||||
{ID: "database-indexes", Label: "Database indexes", Track: "backend-developer"},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("GradeInterviewAnswer error: %v", err)
|
||||
}
|
||||
if grade.AnswerID != "a-1" {
|
||||
t.Fatalf("AnswerID = %q", grade.AnswerID)
|
||||
}
|
||||
if len(grade.Concepts) != 1 {
|
||||
t.Fatalf("concepts = %d, want 1", len(grade.Concepts))
|
||||
}
|
||||
if len(grade.Evidence) != 1 {
|
||||
t.Fatalf("evidence = %d, want 1", len(grade.Evidence))
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user