Agent Architect – Build Multi‑Agent AI Systems

About the role

Pencil is creating an agentic operating system for marketing, combining generative AI with brand‑safe, scalable workflows. As an Agent Architect you will design the core "brain" that powers our creative engine and custom client solutions, turning high‑level creative intent into reliable, multi‑modal AI behavior.

Key responsibilities

• Design and implement multi‑agent workflows, including task decomposition, state management, and tool‑use (RAG, API integration, etc.).
• Develop systematic evaluation frameworks (LLM‑as‑a‑judge, promptfoo, DSPy) to benchmark and improve agent performance at scale.
• Create reusable agents and prompt libraries that enable the platform to serve 1,000+ brands with distinct voices simultaneously.
• Collaborate with AI Engineering and Product teams to decide when to use prompting, retrieval‑augmented generation, or fine‑tuning.
• Lead technical delivery for complex client deployments, translating creative briefs into deterministic AI workflows.

Required profile

• 3+ years of hands‑on experience with generative AI models (LLMs and multimodal).
• Strong systems‑thinking mindset, able to map latent spaces, context windows, and agent interactions.
• Comfortable working with Python, JSON structures, and API documentation.
• Passion for rigorous measurement; experience with A/B testing and benchmarking AI outputs.
• Ability to bridge creative direction and technical implementation.

Required skills

• Python
• JSON
• API integration
• LangChain
• CrewAI
• AutoGen
• GPT‑4, Claude, Gemini
• Stable Diffusion, Midjourney, Video Generation models
• Retrieval‑augmented generation (RAG)
• promptfoo
• DSPy