Unable to use the Java source file corresponding to the class file as context #117
Description
Activity
This is known limitation (even two of them):
ClassFileEditordoesn't useBufferManagerto manage its documentClassFileEditordoesn't work onIFileeither but uses externally saved (extracted) content, seeorg.eclipse.jdt.internal.ui.javaeditor.InternalClassFileEditorInput.getPath()
We might have a solution here, but this would require changes in JDT.
- First: we would need to create the file not somewhere on the disc but in an "invisible" project, so that the file will be "real"
IFile. - After that, wither we use
BufferManagerto manage this file - Or we provide document adapter to IFile, similar to the recent fix for Xtext editors support in LSP4E (see Adapt document to IFile when unable to retrieve file from buffer manager eclipse-lsp4e/lsp4e#1501 & add IAdaptable support to IXtextDocument interface eclipse-xtext/xtext#3615).
Another idea it register the
jdturi schema as a content provider to the CLS. If cls meets such uri, delegate the read action to the IDE side.We already did this for the ABAP support.
Is adding source files from plugin project something that can be realised using https://github.com/kaluchi/jdtbridge ??
This can possibly add this feature request in a much easier wayIs adding source files from plugin project something that can be realised using https://github.com/kaluchi/jdtbridge ?? This can possibly add this feature request in a much easier way
Cool plugin, but it is working at different layer. Maybe it could be used by agents if configured.
Here we are deeper in the mechanics of Copilot/LSP4E communication, which is based on Document/IFile mapping working. If there is a Document but no IFile, LSP4E can't work, and Copilot, which uses it, can't work too.
This comment provides two ways whichshould be possible: #117 (comment).
Yet another possibility, fully generic, which could be implemented by Copilot for every document and every tool: maintain "hidden" IProject and create there temporary files mapped to current document instances, and use these files for LSP4E communication.
Dietrich Travkin (@travkin79) : probably something you could check if you would have time?
Reacted by Dietrich TravkinNote to self: Ok then investigate how eclipse JDT/PDE does it and put in a request there to make it accessible?
Maybe look here https://help.eclipse.org/latest/index.jsp?topic=%2Forg.eclipse.jdt.doc.isv%2Fguide%2Fjdt_api_manip.htmmaintain "hidden" IProject and create there temporary files mapped to current document instances, and use these files for LSP4E communication.
I feel that this is a little bit too heavy. It looks like
Or we provide document adapter to IFileis more suitable to me. Will it have any performance impact btw?And what do you think about: #117 (comment)?
I've checked some code and thought about your suggested approaches:
- Register a content provider for jdt:// URI schema in copilot-for-eclipse (comment Unable to use the Java source file corresponding to the class file as context #117 (comment))
- Tiny change, only in copilot-for-eclipse (
FileUtils.readFileWithStats()), works for JDT class files only, introduced a new (maybe optional) dependency fromcom.microsoft.copilot.eclipse.coretoorg.eclipse.jdt.core
- Tiny change, only in copilot-for-eclipse (
- "Invisible" project with
IFileand BufferManager wiring (comment Unable to use the Java source file corresponding to the class file as context #117 (comment))- Complex change in JDT, likely no or tiny changes in copilot-for-eclipse and LSP4E, works for JDT class files only
- "Invisible" project with
IFileandIAdaptablewiring (comment Unable to use the Java source file corresponding to the class file as context #117 (comment))- Complex change in JDT, lightweight change in copilot-for-eclipse for reading adapters (
Adapters.adapt()), works for JDT, could by used by other plug-ins by also registeringIFileadapters
- Complex change in JDT, lightweight change in copilot-for-eclipse for reading adapters (
- Generic "Invisible" project with
IFilefor anyIDocument(comment Unable to use the Java source file corresponding to the class file as context #117 (comment))- Complex change in copilot-for-eclipse only, works for any
IDocument
- Complex change in copilot-for-eclipse only, works for any
I'd suggest another option that we could take into account:
- Fall back to Eclipse File System (EFS) for reading non-buffered, non-IFile text files
- Register an EFS provider for the jdt:// URI schema and re-use the files created by JDT (without creating hidden
IProjectandIFile) (yes, we still need to adapt JDT, but it's likely a smaller change than maintaining a hiddenIProject).- (An alternative to this could be to handle the jdt:// URI schema directly in copilot-for-eclipse and call JDT code for reading the documents. In this case we would not need any change in JDT at all, but would introduce the same dependency as in approach 1.)
- Adapt copilot-for-eclipse to fall back to
IFileStore/ EFS when reading a file from an URI (handleworkspace/readFileviaFileUtils.readFileWithStats()) (that would work with non-IFile files in file system as well as virtual files like source files linked to binary files, de-compiled files, archived files, remote files, etc. and I guess, it would also work for editors that are not based on LSP4E and the "generic code editor", i.e.org.eclipse.ui.internal.genericeditor.ExtensionBasedTextEditor) - Adapt LSP4E to create URIs also for files outside the Eclipse workspace and for documents whose text file buffer does not map to an
IFile, such asClassFileEditorand Xtext-based editors (LSPEclipseUtils.toUri(IDocument)) by usingIFileStorevia active editor input (IURIEditorInput) - Optional: adapt copilot-for-eclipse to fall back to IFile provided through adapter (
Adapters.adapt()) to catch all other cases (or re-use that logic from LSP4E inLSPEclipseUtils.getFileHandle(URI))
- Register an EFS provider for the jdt:// URI schema and re-use the files created by JDT (without creating hidden
Compared to the approaches 2.-4. we would have relatively small changes in JDT, LSP4E and copilot-for-eclipse, a generic approach in LSP4E and copilot supporting non-text-buffered editors and files outside the Eclipse workspace without any need for hidden
IProjects, avoid introducing JDT dependencies incom.microsoft.copilot.eclipse.core.If we want a short-term solution, we could complement that with approach 1, i.e. register a content provider for jdt:// URI schema in copilot-for-eclipse.
What do you think, Sheng Chen (@jdneo) and Andrey Loskutov (@iloveeclipse)?
- Register a content provider for jdt:// URI schema in copilot-for-eclipse (comment Unable to use the Java source file corresponding to the class file as context #117 (comment))
I've made some experiments and it seems to be more difficult to implement this than I expected.
- JDT does not use a jdt:// URI scheme for it's files and it doesn't use an IURIEditorInput in its ClassFileEditor. Instead, it uses an InternalClassFileEdiorInput that implement IPathEditorInput. That editor input offers a path to a temporary file, but the content seems to be the binary file, not the linked source code.
- I've experimented with ReferencedFileService in copilot to provide access to the source code using (a) an artificial jdt::// URI to the source code, that didn't work, since a file:// URI was expected, so (b) I created a fake file:// URI to the source code, but the code was never called. For some reason, the CLS does not try to read the open class file's source code. It says, it cannot read the source code.
I'm not sure what is the best way to let CLS know that it CAN try reading the source code using some LSP API and the client would provide the source code by calling JDT's code if JDT is available (through optional dependency).
- That editor input offers a path to a temporary file, but the content seems to be the binary file, not the linked source code.
Could you try this below?
- LSP4E in
org.eclipse.lsp4e.LSPEclipseUtils.getDocument(IEditorInput)first would want to get anIFilefromIPathinorg.eclipse.lsp4e.LSPEclipseUtils.getFile(IPath). Let assume we will address that in JDT by creating temporary files not in /tmp as of today but in the hidden project. - Now LSP4E will want to get
IDocumentfromIFileviaorg.eclipse.lsp4e.LSPEclipseUtils.getDocument(IResource). Let assume we would register the buffer with the source code for the class file (which, if I understand it right, is retrieved viaIBuffer org.eclipse.jdt.internal.core.ClassFile.openBuffer(IProgressMonitor, IElementInfo)) viaITextFileBufferManager. If that would work, LSP4E would be able to find theIDocument.
Both changes above would be needed in JDT.
Hi Andrey Loskutov (@iloveeclipse),
I think, reading the file content is not the main issue here. It's more about how reading is triggered by the agent.I did temporarily modify the LSP4E code to return a valid URI, which landed in copilot. (It also worked without modifying LSP4E by adding some JDT-specific code to copilot.) I then was able to add class files to the context in the copilot chat view.
But as soon as I asked copilot something about the file's content, it returned generic answers, saying, it cannot read the file since it is binary and outside the Eclipse workspace. The copilot code for reading the file's content (e.g.
FileUtils.readFileWithStats()) was never called. I guess, it must have something to do with how CLS decides to handle the given file. Maybe there are some internal instructions about how to handle class files or I misunderstood the communication pattern between client and language server.This is why I hoped for some insights from Sheng Chen (@jdneo) on this.
Observations / remarks:
- copilot-for-eclipse excessively relies on
IFileand files within the Eclipse workspace although for LSP URIs are used - Maybe it's worth to focus on document URIs and in addition to
IFilealso considerIFileStore(EFS), i.e. files outside the Eclipse workspace
- copilot-for-eclipse excessively relies on
Thanks for the direction. I’d like to investigate this from the context-resolution boundary rather than just patching the class-file case.
My understanding is that the bug is caused by treating Copilot context as if it must always resolve to a workspace
IFile, while JDT class-file/source-attachment editors can expose valid Java source content without mapping cleanly to a project.javafile.I’ll trace the current path for editor document → context URI → CLS read, compare it with the existing ABAP content-provider mechanism, and check whether a
jdtURI content provider can delegate reads back to the IDE side without changing the normal workspace-file path.If that matches the current architecture, I’ll open a small draft PR with:
- a focused reproduction/validation path,
- minimal implementation around the content-provider boundary,
- and coverage where the repo’s test structure allows it.
This is why I hoped for some insights from Sheng Chen (@jdneo) on this.
The main blocking point is exactly at CLS side. We will send the reference file's uris to CLS, and CLS will try to read them. Since CLS is a node project so it can only read local files, the
file://is the one only allowed. (And those scheme registered via content provider).Thanks, that makes sense.
So the key boundary is not only resolving the editor-side reference, but making sure CLS can read non-
file://references through a registered content provider.I’ll investigate the existing content-provider registration path first and check whether a
jdtscheme provider can delegate the read back to the Eclipse side, keeping normalfile://reads unchanged.That also seems like the smallest first step toward supporting visible external editor documents as read-only context without forcing them into the workspace-file model.
Hello, the platform is: Eclipse 4.39 + Copilot 0.16.0.
When importing a plugin project, if the imported project only contains JAR files and does not include source code, Eclipse can open the class file and display the source files associated with that JAR. However, in Copilot, the source file corresponding to that class file cannot be selected as the AI's context. If Copilot is prompted at this point, it will search for the Java file, but the Java file is not in the project. The output will be:
...
Searched for files matching query: org.eclipse.core.resources/*/.java
Read directory org.eclipse.core.resources
...
In other words, the corresponding Java file cannot be found for analysis. Therefore, the AI cannot read the Java file corresponding to the class file and cannot use it as context for analysis.