| View previous topic :: View next topic |
| Author |
Message |
acheron2 Tux's lil' helper

Joined: 17 Jan 2007 Posts: 100
|
Posted: Sat May 19, 2012 6:16 pm Post subject: Unreproducible errors during compilation |
|
|
I've been using media-video/swfmill with these ebuilds for years on my old computer. New versions compiled without problems, but some people reported compilation errors.
After moving to a new multicore computer I started getting errors too. Odd things are:
1. Compiling the sources in my home directory worked.
2. Compiling the sources with the command | Code: | | ebuild swfmill-0.3.2.ebuild compile | worked.
3. Compiling the sources with the commandfailed several times. Each time with a different error.
What could this be? Some sort of race condition?
When I tried the command | Code: | | MAKEOPTS="-j1" emerge swfmill | it worked. After that compilation started to always work, so I can't reproduce the errors anymore.
If this is a race condition, how can I ascertain that? Or what else could this be? |
|
| Back to top |
|
 |
acheron2 Tux's lil' helper

Joined: 17 Jan 2007 Posts: 100
|
Posted: Sat May 19, 2012 6:24 pm Post subject: |
|
|
Another reason that makes me think it's a race condition, is this. All errors were in *.cpp files generated by xsltproc. All errors were either about unmatched bracket or undeclared variable. When I checked the file, where the error occurred, all brackets were matched and there was no variable with such name, but instead a variable starting with it — no "Trac", but "Trace". As if the compiler read the incomplete source file.
If this is true, how should I change the makefile to avoid it? |
|
| Back to top |
|
 |
Hu Watchman

Joined: 06 Mar 2007 Posts: 7679
|
Posted: Sat May 19, 2012 7:53 pm Post subject: |
|
|
The most likely cause for the behavior you are seeing is that someone wrote a rule of the form: | Code: | a.cpp b.cpp: code.xml.in
: call-to-create-both-outputs | The problem with this is that it is a shorthand for specifying a.cpp and b.cpp as separate targets, so this tells Make to run that command if any of those files is needed. If more than one is needed, Make will run instance of the command for each required file, and one might step on the other. The fix is to identify the set of all source files generated by a single xsltproc call and change the Makefile so that every member of the set depends on a flag file, and runs no commands. Then, make the flag file run the xsltproc command. | Code: | .sources.generated:
: call-to-create-both-C++-files && touch .sources.generated
a.cpp b.cpp: .sources.generated |
|
|
| Back to top |
|
 |
acheron2 Tux's lil' helper

Joined: 17 Jan 2007 Posts: 100
|
Posted: Sat May 19, 2012 10:29 pm Post subject: |
|
|
Thanks.
In Makefile xsltproc is called only once: | Code: |
$(CODEGEN_GENERATEDSOURCES) $(CODEGEN_GENERATEDHEADERS): $(CODEGEN_SRC)
xsltproc $(srcdir)/codegen/mk.xsl $(srcdir)/codegen/source.xml |
The variables are defined as: | Code: |
CODEGEN_GENERATEDSOURCES = \
gSWFParseXML.cpp \
gSWFWriteXML.cpp \
gSWFParser.cpp \
gSWFWriter.cpp \
gSWFDumper.cpp \
gSWFBasics.cpp \
gSWFSize.cpp \
$(NULL)
CODEGEN_GENERATEDHEADERS = \
SWF.h \
$(NULL)
CODEGEN_SRC = \
codegen/basic.xsl \
codegen/basics.xsl \
codegen/dumper.xsl \
codegen/header.xsl \
codegen/mk.xsl \
codegen/parser.xsl \
codegen/parsexml.xsl \
codegen/size.xsl \
codegen/source.xml \
codegen/writer.xsl \
codegen/writexml.xsl \
$(NULL)
|
There are rules that have $(CODEGEN_GENERATEDSOURCES) as a prerequisite (via other intermediate variables) and rules that require only a single generated gSWF*.cpp.
So, what do you suggest? If I replace the top rule with this: | Code: |
$(CODEGEN_GENERATEDSOURCES) $(CODEGEN_GENERATEDHEADERS): .sources.generated
.sources.generated: $(CODEGEN_SRC)
xsltproc $(srcdir)/codegen/mk.xsl $(srcdir)/codegen/source.xml
touch .sources.generated
| will it be enough? (Also, there should be rules to delete .sources.generated during "clean", to not include it in the distributions, etc.)
And another question still remains: how to make sure the errors stopped because of this fix? |
|
| Back to top |
|
 |
Hu Watchman

Joined: 06 Mar 2007 Posts: 7679
|
Posted: Sat May 19, 2012 10:46 pm Post subject: |
|
|
| Yes, the change you describe should work. You can test whether the problem is solved by checking how many times xsltproc is run during compilation. It should run only once. |
|
| Back to top |
|
 |
acheron2 Tux's lil' helper

Joined: 17 Jan 2007 Posts: 100
|
Posted: Sun May 20, 2012 7:33 am Post subject: |
|
|
| Hu wrote: | | You can test whether the problem is solved by checking how many times xsltproc is run during compilation. It should run only once. |
The problem is: it sometimes runs only once with default settings. Even now I don't know, whether -j1 truly solves the problem, or swfmill compilation stopped failing by accident. I did not keep the logs of failed compilations, and now I can't reproduce them
Another question. (No, I am not very familiar with how make works.) For example, xsltproc already started and is in the middle of writing gSWFWriteXML.cpp. Then make tries to execute a rule, that has gSWFWriteXML.cpp as the only prerequisite. What will happen? Will make wait for .sources.generated, or will it start compiling gSWFWriteXML.cpp since it's already present? |
|
| Back to top |
|
 |
Hu Watchman

Joined: 06 Mar 2007 Posts: 7679
|
Posted: Sun May 20, 2012 3:57 pm Post subject: |
|
|
Given the rule you showed above, I believe using -j1 was the reason the failures ceased. For your second question, it depends on other factors. If a single Make is managing the entire process, then I think the order of events would be correct. Make should inventory all of the goals, determine which objects it needs to satisfy those, then determine which sources it needs to satisfy those objects. It should then begin generating those sources, but notice that those sources require the target .sources.generated, so it will build that first. It will then run the null rules defined for the individual sources, then move to compiling. I think the failure mode you describe can occur only if Make delays dependency resolution. You could get into problems with recursive make, since a recursive make may not have all the rules present in a single process.
You could test this by replacing the xsltproc rule with one which writes an ill-formed gSWFWriteXML.cpp, sleeps long enough that other jobs ought to finish, then replaces the ill-formed gSWFWriteXML.cpp with a valid one, and exits successfully. Then run a parallel build and see if it defers the compilations until the sleep terminates. |
|
| Back to top |
|
 |
acheron2 Tux's lil' helper

Joined: 17 Jan 2007 Posts: 100
|
Posted: Sun May 20, 2012 6:43 pm Post subject: |
|
|
Thanks, I'll try that. Probably next weekend  |
|
| Back to top |
|
 |
|