This is what I learned debugging my way from notebooks to production.

The Setup

Goal: Predict if employees will leave + explain why with SHAP.
Stack: Random Forest, FastAPI, Gradio, SHAP

Simple enough. Here's where it broke.

Mistake #1: Missing The Pattern in EDA

During exploratory analysis, I found something crucial: 62% attrition at Year 5.
Employees' satisfaction dropped in Year 4 (0.43), then they left in Year 5. There's a lag—people don't quit when unhappy, they quit after stewing in it.

Also discovered:

• Only 2% get promoted

• Promoted employees rarely leave (95% retention)

• Even high-salary employees leave without career growth

The insight: Attrition follows a predictable career stagnation pattern. Promotions matter more than salary.

This shaped my approach to thresholds and risk levels later.

Mistake #2: The Threshold Trap

Back to my "everyone stays" problem during FastAPI testing.
Model metrics looked perfect: 83% accuracy, good recall/precision, balanced classes. But predictions were always "Stay."

Then I checked probabilities:

prediction_probability = pipeline.predict_proba(input_df)[0]
# [0.52, 0.48]  ← Wait, what?

52% Stay, 48% Leave → Predicts "Stay"

The problem: Sklearn uses 0.5 threshold. To an engineer, 52% is correct. To HR, an employee with 48% leave risk needs attention.

Prediction ≠ Business Risk

I experimented with lower thresholds (0.3-0.4) in notebooks but took a different approach in the API:

prediction = "Leave" if pred == 1 else "Stay"

# But also calculate risk independently
if prediction_probability[1] < 0.4:
    risk_level = 'Low'
elif prediction_probability[1] < 0.7:
    risk_level = 'Medium'
else:
    risk_level = 'High'

Now my API returns:

{
  "prediction": "Stay",
  "leave_probability": 0.48,
  "risk_level": "Medium"
}

The lesson: Model output is technical. Business decisions need context.

Once I sorted this out in the backend, I was ready to build the frontend with Gradio.

Mistake #3: SHAP Integration

SHAP assigns each feature a contribution score. Positive = increases leave risk, negative = decreases it. Simple concept, messy implementation.

Problem 1: Version issues

Tried plotting:

shap.summary_plot(shap_values, features)

Got cryptic NumPy errors. SHAP had a version update that broke old methods.

Solution: Skipped plots, extracted raw values:

# Get SHAP values for class 1 (Leave)
shap_values = explainer(processed_input)
shap_class1 = shap_values.values[0, :, 1]  # Binary classification

# Pair with feature names
feature_names = pipeline.named_steps["preprocess"].get_feature_names_out()
feature_impacts = list(zip(feature_names, shap_class1))

# Sort by absolute impact
top_features = sorted(feature_impacts, key=lambda x: abs(x[1]), reverse=True)[:3]

Problem 2: Making it readable

Raw output: remainder__satisfaction_level: -0.104

HR doesn't care about "remainder__" prefixes. So I cleaned it:

for feature, value in top_features:
    clean_name = feature.split("__")[-1]  # Drop pipeline prefix
    effect = "increases_leave_risk" if value > 0 else "decreases_leave_risk"
    top_factors.append({
        "feature": clean_name,
        "impact": effect,
        "contribution_strength": round(float(value), 3)
    })

Gradio now shows:

↓ satisfaction_level (-0.104)
↑ time_spend_company (0.049)
↓ last_evaluation (-0.101)

The lesson: Explainability is translation, not just computation.

Mistake #4: Path & Host Issues

Paths breaking deployment:

pipeline = joblib.load("experiments/my_pipeline.joblib")

Relative paths depend on where you run the command. Fixed with:

from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent
MODEL_PATH = BASE_DIR.parent / "experiments" / "my_pipeline.joblib"

Host confusion:

Set Gradio to server_name="0.0.0.0". Terminal showed http://0.0.0.0:7860. Clicked it—didn't work.

Browsers can't connect to 0.0.0.0. Use http://127.0.0.1:7860 instead.

Small thing, but it ate 30 minutes of my life.

The Final App

HR enters data, gets:

• Prediction (Stay/Leave)

• Leave probability (10%)

• Risk level (Low/Medium/High)

• Top 3 factors with SHAP values

Clean, actionable, explainable.

What I Learned

Before: Does my model have good accuracy?
After: Does it solve the business problem?

Before: Prediction is correct if it matches the label
After: Prediction ≠ risk. Context matters.

Before: SHAP values are just numbers
After: Explainability is translation for humans

Before: Relative paths work for me
After: Production code must work anywhere

Key Takeaways

1. Domain knowledge shapes everything. Year 5 attrition spike and promotion patterns directly influenced my threshold and risk decisions.

2. Prediction ≠ Business Decision. 48% leave probability might be "Stay" but it's a retention risk HR needs to see.

3. Explainability is for people. SHAP values mean nothing until you translate them into actionable insights.

4. Test in production mode. Paths and hosts that work locally often break in deployment.

5. Systems > Models. Training took hours. Building a usable, explainable API took days.

Final Thoughts

This project taught me more than any tutorial could. Not because I built a perfect system, but because I hit every wall and learned to climb over it.

The threshold trap, SHAP integration struggles, the gap between predictions and business risk—none of this was in any course syllabus. But these are exactly the lessons that separate models that train from systems that matter.

If you're building something similar and hit these same problems, you're not failing. You're learning the parts that actually count.

Got questions? Found better solutions? Hit similar walls? Drop your thoughts in the comments—I'd love to hear your war stories.

View the complete source code and implementation on GitHub

When I first thought about building a resume screener, the idea felt too simple.

Scan the resume.
Scan the job description.
Compute cosine similarity.
Done. Right?

To sanity-check myself, I asked ChatGPT to behave like a senior ML engineer and review the idea. Instead of encouragement, I got slapped with reality:

“Cosine similarity alone won’t work. You need segmentation.”

Rude? Maybe.
Correct? Absolutely.

That single statement turned what I thought was a weekend project into hours of debugging, hair-pulling, and multiple interrogation sessions with GPT. If that sounds familiar, this article is for you.

I’ll walk you through what I built, what broke, and what finally worked, so you can save time — and probably some hair.

The core idea

I split the problem into two independent parts:

1. Resume processing (hard part)

Resumes are messy, inconsistent, and unstructured. So I used a rule-based segmentation approach inspired by TSHD (Topic Segmentation based on Heading Detection).

2. Job Description processing (easier part)

Job descriptions are comparatively structured. For them, sentence embeddings work surprisingly well, so I leaned on that.

This separation alone made the system much easier to reason about.

Step 1: Resume preprocessing

Before any intelligence, I cleaned the text:

• Tokenized the resume

• Removed bullets and punctuation

• Stemmed words (important later)

• Converted the resume into a structure like: (index → sentence)

This preserved order, which is critical for segmentation.

Step 2: Understanding headings (the painful part)

This was the most time-consuming and frustrating part of the project — but also the most important.

Before going into the algorithm, you need to understand two dictionaries.

Cue phrases (multi-word signals)

These capture the different ways people name the same section.

Example: not everyone writes Education. Some write Academic background.

# Example cue phrases (illustrative, not exhaustive)
cue_phrases = {
    "academic background": "education",
    "professional experience": "experience"
}

Think of this as:

“If I see this phrase, I know which section it really means.”

Cue words (single-word signals)

These handle simple, clean headings like Skills or Education.

# Example cue words
cue_words = {
    "education": "education",
    "experience": "experience"
}

Important detail:

• These words are stemmed internally.

• So education, educational, and educ all collapse to the same signal.

Step 3: Heading detection (Scan-1 and Scan-2)

This is where most things broke before they worked.

Scan-1: Bigram detection

• Create bigrams from each resume line

• If a bigram matches a cue phrase:

  • Store the section name

  • Store the line index

  • Add it to a heading_document

Scan-2: Single-word detection

• Look at short lines (likely headings)

• If a token matches a cue word and isn’t already detected:

  • Add it to the heading document

By the end of this step, I had:

{ section_name → starting_index }

Important rule:
Cue phrases, cue words, and resume text must be in the same processed format, or nothing works.

Step 4: Resume segmentation

Once headings and their start indices are known:

1. Sort them by index

2. Slice the resume between consecutive headings

This gives clean sections like:

• summary

• experience

• projects

• skills

• education

At this point, the resume is finally structured.

Step 5: Job Description segmentation (semantic approach)

Instead of relying on JD headings, I took a different route.

I created semantic templates, like:

• “This section describes responsibilities”

• “This section lists mandatory skills”

Each JD sentence was embedded and compared against these templates, then bucketed into:

• role overview

• responsibilities

• required skills

• nice to have

This made the system robust to poorly formatted JDs.

Step 6: Matching resume ↔ JD

Each resume section and JD bucket was embedded.

Then I applied a controlled matching policy, for example:

• JD required skills → resume skills, experience, projects

• JD responsibilities → resume experience, projects

For each JD bucket, I took the maximum similarity across allowed resume sections.

Finally, I computed a weighted score, prioritizing required skills and responsibilities.

What I learned (this matters more than the code)

• Always be clear about intermediate outputs, not just the final score

• Don’t implement everything at once — build and validate step by step

• Move forward only when the current step makes sense

• Before coding, write the logic on paper and explain it to yourself

It sounds simple. It isn’t. But it works.

Final thoughts

If you’ve reached this point, you’re more than capable of building your own resume screening system.

If you’re still unsure, don’t worry — I’ll link:

• My GitHub repository

• The research paper that inspired this approach

Build it. Break it. Learn from it.
And when you’re done, come back and tell me what you learned.

Happy coding 🚀