🐛 Bug Report: Langchain instrumentation adds empty create_agent graph nodes for every middleware hook #4628

Activity

1. DEVJHAWAR11 commented on Oct 16, 2026

   

   DEVJHAWAR11

   on Oct 16, 2026

   More actions

   Hi, I can reproduce this with the script above. The base hooks get wrapped, so AgentMiddleware.before_model is AgentMiddleware.before_model turns False and create_agent adds a node per middleware. I'd like to fix it. #4000 looks like the same idea but has been conflicting since April. Would you prefer I rebase and finish that one, or open a fresh PR with a regression test (graph nodes unchanged after instrument(), and the hooks still traced when a middleware really overrides them)? Happy to go either way.
2. scampbell-h1 commented on Oct 16, 2026

   

   scampbell-h1

   on Oct 16, 2026

   Author

   More actions

   One more effect of the same wrapping, which a colleague spotted: it can stop a middleware's hook from running at all.

   A middleware hook has a sync and an async version. If a middleware writes only one of them, LangChain normally either runs that one or raises an error. With instrumentation on, LangChain thinks both were written, so it runs the wrapped empty base method in place of the missing one. The hook is skipped silently.

   Middleware defines	Agent called with	No tracing	0.62.4 instrumented
   only before_model	ainvoke	hook runs	hook silently skipped
   only abefore_model	invoke	raises TypeError	hook silently skipped

   Reproduction steps

   1. Save the script below as repro_dispatch.py.
   2. Run it in a clean environment:

      uv run --no-project --with "opentelemetry-instrumentation-langchain==0.62.4" --with "langchain>=1.3" --with opentelemetry-sdk python repro_dispatch.py
      

   import asyncio
   
   from langchain.agents import create_agent
   from langchain.agents.middleware import AgentMiddleware
   from langchain_core.language_models.fake_chat_models import FakeMessagesListChatModel
   from langchain_core.messages import AIMessage
   
   
   class SyncOnly(AgentMiddleware):
       calls = 0
   
       def before_model(self, state, runtime):
           SyncOnly.calls += 1
   
   
   class AsyncOnly(AgentMiddleware):
       async def abefore_model(self, state, runtime):
           return None
   
   
   def agent(middleware):
       model = FakeMessagesListChatModel(responses=[AIMessage(content="done")])
       return create_agent(model, tools=[], middleware=[middleware])
   
   
   def dispatch():
       request = {"messages": [{"role": "user", "content": "hi"}]}
       SyncOnly.calls = 0
       asyncio.run(agent(SyncOnly()).ainvoke(request))
       try:
           agent(AsyncOnly()).invoke(request)
           async_only = "ran, no error"
       except Exception as error:
           async_only = f"raised {type(error).__name__}"
       return f"sync-only under ainvoke: called {SyncOnly.calls}x | async-only under invoke: {async_only}"
   
   
   print("before instrument:", dispatch())
   
   from opentelemetry.instrumentation.langchain import LangchainInstrumentor
   
   LangchainInstrumentor().instrument()
   print("after instrument: ", dispatch())

   Expected

   Instrumenting doesn't change which hooks run:

   before instrument: sync-only under ainvoke: called 1x | async-only under invoke: raised TypeError
   after instrument:  sync-only under ainvoke: called 1x | async-only under invoke: raised TypeError
   

   Actual (0.62.4, langchain 1.4.6, Python 3.14.0)

   before instrument: sync-only under ainvoke: called 1x | async-only under invoke: raised TypeError
   after instrument:  sync-only under ainvoke: called 0x | async-only under invoke: ran, no error
   
3. DEVJHAWAR11 commented on Oct 16, 2026

   

   DEVJHAWAR11

   on Oct 16, 2026

   More actions

   I opened a PR with a fix: #4629. The base hooks are now plain functions via functools.wraps, so the identity check, the runtime signature and the sync/async dispatch all match the un-instrumented behaviour. Thanks @scampbell-h1 for the dispatch repro, it's in the tests. This should replace #4000, which fails with the runtime error described above.
4. sdelmas commented on Oct 18, 2026

   

   sdelmas

   on Oct 18, 2026

   More actions

   I can confirm the skipped hook from the comment above with a standalone repro (scripted chat model, no API key). Versions: langchain 1.3.14, langchain-core 1.4.9, langgraph 1.2.12, opentelemetry-sdk 1.45.1.

   • Sync-only before_model (subclass or @before_model) on ainvoke: runs 1/1 uninstrumented. On 0.62.1, 0.62.4 and main 94d3f04 it runs 0/1 and raises no error. With the fix(langchain): keep AgentMiddleware hook identity when instrumenting #4629 head (2d0c82c) it runs 1/1.
   • Async-only abefore_model (subclass or @before_model) on invoke: uninstrumented, it raises TypeError: No synchronous function provided. On 0.62.1, 0.62.4 and main it runs 0/1 with no error. With fix(langchain): keep AgentMiddleware hook identity when instrumenting #4629 it raises the same TypeError.

   In the skipped runs an execute_task <Middleware>.abefore_model (or .before_model) span still appears. It comes from the wrapped base no-op, which runs in place of the user's hook.

   With #4629 installed, every instrumented case matches the uninstrumented one. One side effect: these middlewares no longer emit any execute_task hook span, because no base hook runs. That might be worth a changelog note.

   Repro: create_agent with one middleware and a fake chat model that answers immediately. Each case runs in a fresh process, once with LangchainInstrumentor().instrument() and once without, and counts hook calls against model calls.