It's been kinda hard to get Eclipse to work well with big CDT projects (say, the Linux kernel, or even a BuildRoot project including the Linux kernel). The documentation for Eclipse is between non-existent and unhelpful, and I only could find a post that talked specifically about preparing an Eclipse project for the Linux kernel. But that post explains nothing, and does some arbitrary things. So this is the generalization of that.
The high-level steps are quite simple (I'm using Eclipse Luna SR2):
It might happen that if the makefile changes working directories during operation and the include directories are relative, then the GBOP gets lost and misses those directories. In this case, included files and include folders can be specified manually with the “symbols and paths” sheet in the project properties.
Now, everything is configured, you run a clean and build of the project, ... and now what? Did it work? How to know?
If no errors appear anywhere, probably that means that everything worked. Congrats, you're finished!
But in my case I usually find that there are lots of symbols that Eclipse considers undefined, even though the build was successful. So, let's fix that.
First of all: what went wrong? Since the feedback is preciously little, let's start by maximizing it. Go to Eclipse Preferences - General - Appearance - Label Decorations, and make sure that the following checkboxes are checked:
Another way to maximize the feedback is setting the GBOP to parse file-wise, at the very least until we have a well-working index. That way every detected file will be decorated, and the missed files will stand out. In any case, note that the decorations might not update when you expect them to; if things don't seem to change when they should, try Project - C/C++ Index - Refresh, or even restart the whole Eclipse. This has helped me a worrying number of times...
You can add the dir with build products to the output directories properties, but (according to Mozilla's instructions for their developers) that is only for debugging. Also, in the source directories one can filter the non-wanted source folders, or alternatively delete the base folder and only add the wanted folder. Looks like the result is the same, but filtering is more visible in the GUI.
Some gotchas:
BuilRoot specifics
Turns out that BuildRoot changes some details of how the Linux build is done, which requires extra caring in Eclipse. A normal Linux build generates the object files right where the sources are, plus some include files are generated into the directory include/generated (and various other generated directories, like in arch; plus for example kernel/timeconst.h). In contrast, BuildRoot builds Linux adding an -O option to the make process, and so puts all the build products in a parallel tree; for Linux this is on <buildroot root>/output/linux*. And the generated include files are generated in there! So Eclipse has to be configured to also look for include files there.
More specifically, the include, include/uapi, arch/include, arch/include/uapi dirs (relative to the kernel root) were in my case caught by the GBOP; but the versions of those dirs in <buildroot root> output/build/linux-custom/* had to be added manually.
So while I'd guess that the standard Linux kernel build process might be almost automatically managed by the Eclipse detectors once minimally configured, the BuildRoot case is much harder. By following the BuildRoot build process through strace, one can see that the generated include dir paths are being tried against what seems to be the configured working directory for make. The heuristics don’t really help because they tend to convert the negatives into false positives, like including files for another architecture - even if that architecture has been manually excluded from the build through Eclipse’s source location filters.
strace -f -e trace=file -p $ECLIPSE_PID shows the paths being tried and/or actually used. Very verbose and gritty/slowish, but is the “ground-truth” of what Eclipse can see. Add a 2>&1 | grep ENOENT (to get the nonexisting files) or | grep generated (to get the paths tried) for some sort of real-time path problem alert. For anything more complicated, better just dump to a file with -o myfile and examine it after the fact.
Right click on project - Index - Find unresolved includes: gives a quick comfortable look on where the problems are, once a full build is done. But making the full build is slow itself, so iterating on this can easily end up being slower than getting the strace log. (For the record, I don’t know what do exactly do the “rebuild index” and “re-resolve unresolved includes” options, but the times I have tried them they didn’t change much the index coverage from the last clean+build)
Right click on file - Index - Create indexer log: shows everything the indexer knows about the file, and the problems in it; particularly at the end it shows the unresolved includes. However it only summarizes what is not working, doesn’t tell you why. For that, we need to get inventive.
For example, with a classic find . -name 'unresolved_include.h' we can find where there are files named like those unresolved includes, so we can know which real files we should be targetting with the Eclipse configuration. And with the strace we can see what Eclipse is trying to do with its existing configuration.
A nice oneliner to get a sorted, deduplicated view of the files failing to resolve: first take the strace log of the build process; then,
cat eclipsebuild.stracelog | grep ENOENT | sort -k2 | uniq -s5 -d -c
[1] It is also possible to abuse the GBISD to make it do part of the work from step 3: just put in the detector’s command line the params from the normal gcc command line used during the build. However, these detected settings will be “global” for the configuration, so beware. Abusing the GBISD is just nice in that it shows clearly what was detected, but kinda risky because of the global assumption. If you are using anything different than “project” in the GBOP, then probably you shouldn’t use this trick.
The high-level steps are quite simple (I'm using Eclipse Luna SR2):
- Find out what is needed in your command-line build process to turn it verbose. We want the full, individual gcc (or clang or whatever) command lines to be shown. Probably something like adding "V=1" to a make command line; though sometimes the makefile is already verbose and nothing is needed.
- Dissect that command-line invocation you just used into the pieces needed for Eclipse's project's "C/C++ Build" properties: build command (probably "make"), working dir, plus the "clean" and "build" targets. Fill them in. Once again, the defaults might be good already.
- Fill-in the command-line call for the GCC built-in specs detector (let's call it GBISD), which is inside the "C/C++ General - Preprocessor Include Paths, Macros, etc" settings. Those detected built-ins will then be used by Eclipse to parse the sources, so you need to be aware of the interplay betwen sources - built-ins - build flags. For example: if the sources are being built with -std=c99, GCC sets its built-in #defines accordingly. Eclipse will get those #defines through the GBISD, so we have to provide the GBISD with the -std=c99 flag too. Once the normal building happens, Eclipse will NOT be paying attention to flags like that -std=c99. Other flags needing this treatment: -nostdinc, -freestanding, etc
- Fill-in the compiler name in the GCC build output parser (let's call it GBOP), also in the "Preprocessor Include Paths, etc" section.
- Finally, you could try filtering non-interesting sources. For example, I was only interested on editing a driver, so I could filter the rest and stop Eclipse from indexing tons of unrelated stuff – which would have taken hours, at best.
It might happen that if the makefile changes working directories during operation and the include directories are relative, then the GBOP gets lost and misses those directories. In this case, included files and include folders can be specified manually with the “symbols and paths” sheet in the project properties.
Now, everything is configured, you run a clean and build of the project, ... and now what? Did it work? How to know?
If no errors appear anywhere, probably that means that everything worked. Congrats, you're finished!
But in my case I usually find that there are lots of symbols that Eclipse considers undefined, even though the build was successful. So, let's fix that.
First of all: what went wrong? Since the feedback is preciously little, let's start by maximizing it. Go to Eclipse Preferences - General - Appearance - Label Decorations, and make sure that the following checkboxes are checked:
- C/C++ Files and Folders with Customized Build Settings. This places a small wrench icon (to me it rather looks like a key) on the top right of the icon of folders/files which have custom build settings... like those captured by GBOP!
- C/C++ Indexed Files - this places a small green dot on the top left of the icon of files that have been indexed.
Another way to maximize the feedback is setting the GBOP to parse file-wise, at the very least until we have a well-working index. That way every detected file will be decorated, and the missed files will stand out. In any case, note that the decorations might not update when you expect them to; if things don't seem to change when they should, try Project - C/C++ Index - Refresh, or even restart the whole Eclipse. This has helped me a worrying number of times...
You can add the dir with build products to the output directories properties, but (according to Mozilla's instructions for their developers) that is only for debugging. Also, in the source directories one can filter the non-wanted source folders, or alternatively delete the base folder and only add the wanted folder. Looks like the result is the same, but filtering is more visible in the GUI.
Some gotchas:
- even though the filter-definition dialog allows to filter out a folder and leave enabled its subfolders, in practice the subfolders get filtered too. So the full path to any enabled subfolder must traverse only enabled folders.
- a folder filtered out for sources will also block binaries from being found in the same directory – sounds like a bug to me.
- the heuristics options in the indexer and GBOP provider seem to be less than helpful, in that when I disabled them an unresolved include started resolving. In fact the only way in which I could get a perfect result was by using the dumbest settings: no indexing of unbuilt sources, no heuristics. Also, the extra work caused by the heuristics seem to make the build much slower – about 3 times slower IIRC.
- for multi-makefile projects in which each makefile can be defining its own environment variables, there are in principle 2 ways to attack the problem of getting the Eclipse build to do everything it should:
- to reimplement the build mechanism inside of the Eclipse project properties (doomed to fail and/or waste a lot of time, plus the brittleness of having to keep the Eclipse settings synced to what the Makefiles do), or
- find a way (write a script or root makefile?) to run the whole build from a single command-line command, which will be used by Eclipse, and let Eclipse deduce everything from the compile-time command lines and messages and generated binaries.
BuilRoot specifics
Turns out that BuildRoot changes some details of how the Linux build is done, which requires extra caring in Eclipse. A normal Linux build generates the object files right where the sources are, plus some include files are generated into the directory include/generated (and various other generated directories, like in arch; plus for example kernel/timeconst.h). In contrast, BuildRoot builds Linux adding an -O option to the make process, and so puts all the build products in a parallel tree; for Linux this is on <buildroot root>/output/linux*. And the generated include files are generated in there! So Eclipse has to be configured to also look for include files there.
More specifically, the include, include/uapi, arch/
So while I'd guess that the standard Linux kernel build process might be almost automatically managed by the Eclipse detectors once minimally configured, the BuildRoot case is much harder. By following the BuildRoot build process through strace, one can see that the generated include dir paths are being tried against what seems to be the configured working directory for make. The heuristics don’t really help because they tend to convert the negatives into false positives, like including files for another architecture - even if that architecture has been manually excluded from the build through Eclipse’s source location filters.
Some tools for easing the pain
Until the docs for the Eclipse/CDT indexer get to be good enough, there is going to be a lot of guesswork needed. So the most important thing right now is to have some tools to help guess right.strace -f -e trace=file -p $ECLIPSE_PID shows the paths being tried and/or actually used. Very verbose and gritty/slowish, but is the “ground-truth” of what Eclipse can see. Add a 2>&1 | grep ENOENT (to get the nonexisting files) or | grep generated (to get the paths tried) for some sort of real-time path problem alert. For anything more complicated, better just dump to a file with -o myfile and examine it after the fact.
Right click on project - Index - Find unresolved includes: gives a quick comfortable look on where the problems are, once a full build is done. But making the full build is slow itself, so iterating on this can easily end up being slower than getting the strace log. (For the record, I don’t know what do exactly do the “rebuild index” and “re-resolve unresolved includes” options, but the times I have tried them they didn’t change much the index coverage from the last clean+build)
Right click on file - Index - Create indexer log: shows everything the indexer knows about the file, and the problems in it; particularly at the end it shows the unresolved includes. However it only summarizes what is not working, doesn’t tell you why. For that, we need to get inventive.
For example, with a classic find . -name 'unresolved_include.h' we can find where there are files named like those unresolved includes, so we can know which real files we should be targetting with the Eclipse configuration. And with the strace we can see what Eclipse is trying to do with its existing configuration.
A nice oneliner to get a sorted, deduplicated view of the files failing to resolve: first take the strace log of the build process; then,
cat eclipsebuild.stracelog | grep ENOENT | sort -k2 | uniq -s5 -d -c
[1] It is also possible to abuse the GBISD to make it do part of the work from step 3: just put in the detector’s command line the params from the normal gcc command line used during the build. However, these detected settings will be “global” for the configuration, so beware. Abusing the GBISD is just nice in that it shows clearly what was detected, but kinda risky because of the global assumption. If you are using anything different than “project” in the GBOP, then probably you shouldn’t use this trick.
Comments
Post a Comment