View previous topic :: View next topic |
Author |
Message |
nsoveiko n00b
Joined: 04 Dec 2004 Posts: 53 Location: somewhere over the rainbow
|
Posted: Wed Aug 29, 2012 8:53 pm Post subject: Segmentation faults in various binaries |
|
|
my desktop box started throwing segmentation faults today. they are repeatable and always happen at the same spot.
first it was firefox (strace output):
Code: | ...
open("/lib/libuuid.so.1", O_RDONLY|O_CLOEXEC) = 4
read(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200\22\0\0004\0\0\0"..., 512) = 512
fstat64(4, {st_mode=S_IFREG|0755, st_size=17924, ...}) = 0
mmap2(NULL, 20708, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0xb3a12000
mmap2(0xb3a16000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x3) = 0xb3a16000
close(4) = 0
mprotect(0xb3a16000, 4096, PROT_READ) = 0
mprotect(0xb3a18000, 31465472, PROT_READ|PROT_WRITE) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x6c512215} ---
+++ killed by SIGSEGV +++
Segmentation fault
Exit 139 |
/lib/libuuid.so.1 is owned by sys-apps/util-linux. reemerged it to no avail. reemerging firefox fails on install stage:
Code: | ...
adding: hyphenation/hyph_af.dic (deflated 55%)
/bin/sh: line 1: 26135 Segmentation fault /var/tmp/portage/www-client/firefox-10.0.6/work/mozilla-esr10/obj-i686-pc-linux-gnu/dist/bin/run-mozilla.sh /var/tmp/portage/www-client/firefox-10.0.6/work/mozilla-esr10/obj-i686-pc-linux-gnu/dist/bin/xpcshell -g "$PWD" -a "$PWD" -f /var/tmp/portage/www-client/firefox-10.0.6/work/mozilla-esr10/toolkit/mozapps/installer/precompile_cache.js -e "populate_startupcache('GreD', 'omni.ja', 'startupCache.zip');"
make[1]: *** [install] Error 139
make[1]: Leaving directory `/var/tmp/portage/www-client/firefox-10.0.6/work/mozilla-esr10/obj-i686-pc-linux-gnu/browser/installer'
make: *** [install] Error 2
emake failed |
this is repeatable, and fails at the same point.
other binaries that are segfaulting:
chromium
Code: | ...
open("/usr/lib/libhogweed.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\313zL4\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=68752, ...}) = 0
mmap2(NULL, 70164, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xaf46a000
mmap2(0xaf47a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xf) = 0xaf47a000
close(3) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xaf469000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xaf468000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xaf467000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xaf466000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xaf465000
set_thread_area({entry_number:-1 -> 6, base_addr:0xaf465a80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xb1938000, 8192, PROT_READ) = 0
mprotect(0xaf504000, 4096, PROT_READ) = 0
mprotect(0xaf4d7000, 4096, PROT_READ) = 0
mprotect(0xaf47a000, 4096, PROT_READ) = 0
mprotect(0xaf515000, 4096, PROT_READ) = 0
mprotect(0xb429d000, 4096, PROT_READ) = 0
mprotect(0xb2ddf000, 4096, PROT_READ) = 0
mprotect(0xaf518000, 31465472, PROT_READ|PROT_WRITE) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x68012215} ---
+++ killed by SIGSEGV +++
Segmentation fault
Exit 139 |
open office calc:
Code: | ...
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb772f728) = 27663
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x807f500, [], 0}, {SIG_DFL, [], 0}, 8) = 0
waitpid(-1, /usr/lib/openoffice/program/soffice: line 122: 27673 Segmentation fault "$sd_prog/$sd_binary" "$@"
[{WIFEXITED(s) && WEXITSTATUS(s) == 139}], 0) = 27663
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=27663, si_status=139, si_utime=0, si_stime=0} ---
... |
how do i troubleshoot this thing? |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9675 Location: almost Mile High in the USA
|
Posted: Wed Aug 29, 2012 11:34 pm Post subject: |
|
|
Did you compile a new kernel or build something recently?
Was it working just fine before?
Hardware issues? overheating?
Bad power? Checked if RAM is still good (run memtest86+ ?) _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
nsoveiko n00b
Joined: 04 Dec 2004 Posts: 53 Location: somewhere over the rainbow
|
Posted: Wed Aug 29, 2012 11:49 pm Post subject: |
|
|
eccerr0r wrote: | Did you compile a new kernel or build something recently?
Was it working just fine before? |
i've upgraded kernel (3.2.12 -> 3.3.8 ) and glibc (2.14.1-r3 -> 2.15-r2) yesterday, but both firefox and oocalc had been working just fine after that.
Quote: | Hardware issues? overheating?
Bad power? Checked if RAM is still good (run memtest86+ ?) |
i don't think it's a hardware problem. segfaults are 100% repeatable and always happen at exactly the same points.
Last edited by nsoveiko on Thu Aug 30, 2012 2:09 am; edited 2 times in total |
|
Back to top |
|
|
The Doctor Moderator
Joined: 27 Jul 2010 Posts: 2678
|
Posted: Thu Aug 30, 2012 12:00 am Post subject: |
|
|
revdep-rebuild? _________________ First things first, but not necessarily in that order.
Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box. |
|
Back to top |
|
|
nsoveiko n00b
Joined: 04 Dec 2004 Posts: 53 Location: somewhere over the rainbow
|
Posted: Thu Aug 30, 2012 12:03 am Post subject: |
|
|
penguin swordmaster wrote: | revdep-rebuild? |
done that. |
|
Back to top |
|
|
Ant P. Watchman
Joined: 18 Apr 2009 Posts: 6920
|
Posted: Thu Aug 30, 2012 2:02 am Post subject: |
|
|
Oddly enough I've been getting lscpu (also from util-linux) segfaulting recently. I've done an emerge -e system already. |
|
Back to top |
|
|
nsoveiko n00b
Joined: 04 Dec 2004 Posts: 53 Location: somewhere over the rainbow
|
Posted: Thu Aug 30, 2012 9:33 pm Post subject: |
|
|
i bit the bullet and did 'emerge -e system' today as well. didn't help.
so far these binaries are confirmed to be affected:
Code: | WARNING: Segmentation Faults in these executables
Gdk-2.0 : 1 Time(s)
chrome : 2 Time(s)
conftest : 1 Time(s)
firefox : 3 Time(s)
gdk-pixbuf-quer : 1 Time(s)
ld-linux.so.2 : 1 Time(s)
mythbackend : 3 Time(s)
mythfilldatabas : 7 Time(s)
nepomukservices : 1 Time(s)
nvidia-settings : 3 Time(s)
rsvg-convert : 4 Time(s)
sensord : 2 Time(s)
soffice.bin : 1 Time(s)
xpcshell : 3 Time(s)
|
|
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9675 Location: almost Mile High in the USA
|
Posted: Thu Aug 30, 2012 10:05 pm Post subject: |
|
|
There was someone else complaining about these segfaults, I have no idea why some people are having problems with it and some that are not.
Currently I don't see segfaults on either of my x86_64 machines:
(my laptop)
Linux mikuru 3.4.9-gentoo #2 SMP Sun Aug 26 09:37:03 MDT 2012 x86_64 Intel(R) Core(TM) i5-3317U CPU @ 1.70GHz GenuineIntel GNU/Linux
gcc (Gentoo 4.5.4 p1.0, pie-0.4.7) 4.5.4
(my mythtv box)
Linux ouka 3.2.12-gentoo #2 SMP Mon May 28 21:45:53 MDT 2012 x86_64 Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz GenuineIntel GNU/Linux
gcc (Gentoo 4.5.3-r2 p1.1, pie-0.4.7) 4.5.3
Both are running:
sys-libs/glibc-2.15-r2:2.2
I still can't wait to replicate the problem... this is too strange for me currently... |
|
Back to top |
|
|
nsoveiko n00b
Joined: 04 Dec 2004 Posts: 53 Location: somewhere over the rainbow
|
Posted: Thu Aug 30, 2012 10:59 pm Post subject: |
|
|
eccerr0r wrote: | There was someone else complaining about these segfaults, I have no idea why some people are having problems with it and some that are not.
Currently I don't see segfaults on either of my x86_64 machines: |
for the record, my setup is x86 32 bit
Code: | Portage 2.1.11.9 (default/linux/x86/10.0/desktop/kde, gcc-4.5.3, glibc-2.15-r2, 3.3.8-gentoo-ns i686)
=================================================================
System uname: Linux-3.3.8-gentoo-ns-i686-AMD_Phenom-tm-_9550_Quad-Core_Processor-with-gentoo-2.1
Timestamp of tree: Mon, 27 Aug 2012 11:15:01 +0000
app-shells/bash: 4.2_p37
dev-java/java-config: 2.1.11-r3
dev-lang/python: 2.7.3-r2, 3.2.3
dev-util/cmake: 2.8.8-r3
dev-util/pkgconfig: 0.27
sys-apps/baselayout: 2.1-r1
sys-apps/openrc: 0.9.8.4
sys-apps/sandbox: 2.5
sys-devel/autoconf: 2.13, 2.68
sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.6
sys-devel/binutils: 2.22-r1
sys-devel/gcc: 4.3.4, 4.4.5, 4.5.3-r2
sys-devel/gcc-config: 1.7.3
sys-devel/libtool: 2.4-r1
sys-devel/make: 3.82-r3
sys-kernel/linux-headers: 3.1 (virtual/os-headers)
sys-libs/glibc: 2.15-r2
Repositories: gentoo x-local
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="*"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/share/config /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /usr/share/X11/xkb"
CXXFLAGS="-O2 -march=native -pipe -fomit-frame-pointer"
DISTDIR="/var/lib/portage/distfiles"
EMERGE_DEFAULT_OPTS="--jobs 4 --load-average 8 --keep-going"
FCFLAGS="-O2 -march=i686 -pipe"
FEATURES="assume-digests binpkg-logs clean-logs collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parallel-install parse-eapi-ebuild-head protect-owned sandbox sfperms split-elog split-log strict suidctl unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -march=i686 -pipe"
GENTOO_MIRRORS="ftp://mirror.csclub.uwaterloo.ca/gentoo-distfiles/ http://mirror.csclub.uwaterloo.ca/gentoo-distfiles/ http://mirror.the-best-hosting.net"
LANG="en_CA.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en"
MAKEOPTS="-j 8 -l 8"
PKGDIR="/var/lib/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/var/lib/portage/tree"
PORTDIR_OVERLAY="/var/lib/portage/local"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 aac acl acpi alsa amarok amr apache2 artswrappersuid asf audiofile bash-completion berkdb binary-drivers bittorrent bittorrent-external blas branding bzip2 cairo caps cdda cddb cdparanoia cdr cdrom cli colordiff consolekit cracklib crypt css cups curl cxx dbus declarative device-mapper dga divx djvu dmx dri dts dv dvb dvd dvdnav dvdr enca encode exif fam fbcon ffmpeg firefox flac foomaticdb fortran ftp gd gdbm geoip gif gimp gphoto2 gpm gs gtk iconv id3 id3tag idn imagemagick java javascript jfs jpeg kde kdehiddenvisibility kipi kpathsea latex lcms lensfun libnotify lm_sensors logrotate logwatch lzo mad matroska mbox md5sum memlimit mime mjpeg mmx mmxext mng modules mozilla mp2 mp3 mp4 mpeg mpeg2 mplayer mtp mudflap multiuser musicbrainz mysql na_dd na_icons ncurses network network-cron nls nptl nsplugin nvidia offensive ogg openal opengl openmp pam pango pch pcre pdf phonon plasma png pnm policykit posix ppds pppd pvr qt3support qt4 quicktime radio rar readline realmedia reiserfs rrdcgi rss sdl secure-delete semantic-desktop session sharedmem shorten slang smp sndfile sockets sox speex spell sql srt sse sse2 ssl startup-notification svg system-sqlite tcpd tex4ht theora threads threadsafe tiff truetype tv_check udev udisks umfpack unicode upower usb v4l v4l2 vcd vdpau vim vim-pager vim-syntax vim-with-x visualization vorbis webkit webmail win32codecs wma wmf wmp wordperfect wxwidgets x264 x86 xanim xcb xcomposite xine xml xmp xosd xscreensaver xv xvid xvmc zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="canon ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" PHP_TARGETS="php5-3" PYTHON_TARGETS="python3_2 python2_7" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="nvidia vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
|
|
|
Back to top |
|
|
The Doctor Moderator
Joined: 27 Jul 2010 Posts: 2678
|
Posted: Thu Aug 30, 2012 11:48 pm Post subject: |
|
|
nsoveiko wrote: | i bit the bullet and did 'emerge -e system' today as well. didn't help.
so far these binaries are confirmed to be affected:
Code: | WARNING: Segmentation Faults in these executables
Gdk-2.0 : 1 Time(s)
chrome : 2 Time(s)
conftest : 1 Time(s)
firefox : 3 Time(s)
gdk-pixbuf-quer : 1 Time(s)
ld-linux.so.2 : 1 Time(s)
mythbackend : 3 Time(s)
mythfilldatabas : 7 Time(s)
nepomukservices : 1 Time(s)
nvidia-settings : 3 Time(s)
rsvg-convert : 4 Time(s)
sensord : 2 Time(s)
soffice.bin : 1 Time(s)
xpcshell : 3 Time(s)
|
|
Have you tried world as well? The affected binaries are all part of your world set.
I also noticed that you have FCFLAGS="-O2 -march=i686 -pipe"
while your CXXFLAGS="-O2 -march=native -pipe -fomit-frame-pointer". The -march options do not match. I honestly have no idea if this is a problem or not. Its just grasping at straws. _________________ First things first, but not necessarily in that order.
Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9675 Location: almost Mile High in the USA
|
Posted: Thu Aug 30, 2012 11:53 pm Post subject: |
|
|
Oh... 32-bit x86...
All of my segfault problems ended up being bad hardware.
All of them.
x86 has been fairly stable for me... You should double check your hardware, run memtest86+ and make sure it does not report bad RAM. One of my x86 machines had a weird problem that it was randomly segfaulting - it turned out that its CMOS got corrupted somehow, and configured the chipset/RAM wrong. Resetting the CMOS fixed the problem -- and I'd definitely call this a hardware issue...
This was the thread I was thinking about https://forums.gentoo.org/viewtopic-t-920108.html ... and not solved AFAIK. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
nsoveiko n00b
Joined: 04 Dec 2004 Posts: 53 Location: somewhere over the rainbow
|
Posted: Fri Aug 31, 2012 12:39 am Post subject: |
|
|
penguin swordmaster wrote: | Have you tried world as well? The affected binaries are all part of your world set. |
i call 'emerge -e world' doing it the hard way it actually might be easier to reinstall the system. it's just my primary machine and i can't afford at the moment to shut it down for a couple of days.
there's gotta be a better way...
Quote: | I also noticed that you have FCFLAGS="-O2 -march=i686 -pipe"
while your CXXFLAGS="-O2 -march=native -pipe -fomit-frame-pointer". The -march options do not match. I honestly have no idea if this is a problem or not. Its just grasping at straws. |
FFLAGS and FCFLAGS are fortran compiler flags. they're not set in my make.conf and are inherited from whatever the defaults are. |
|
Back to top |
|
|
nsoveiko n00b
Joined: 04 Dec 2004 Posts: 53 Location: somewhere over the rainbow
|
Posted: Fri Aug 31, 2012 4:23 pm Post subject: |
|
|
eccerr0r wrote: | x86 has been fairly stable for me... You should double check your hardware, run memtest86+ and make sure it does not report bad RAM. One of my x86 machines had a weird problem that it was randomly segfaulting - it turned out that its CMOS got corrupted somehow, and configured the chipset/RAM wrong. Resetting the CMOS fixed the problem -- and I'd definitely call this a hardware issue... |
i've run memtest86+ overnight, two full tests completed, no errors. reset CMOS just in case. nothing. the only thing this has accomplished is that X11 is now segfaulting as well. now i'm officially hosed |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9675 Location: almost Mile High in the USA
|
Posted: Fri Aug 31, 2012 6:36 pm Post subject: |
|
|
Does the livecd segfault too?
Other OS?
What if you mount and chroot into the livecd and run apps there?
Did you try reverting to the old kernel?
Can you try underclocking your CPU to rule that out as a possible issue? Make sure it's not overheating?
Might want to try a reinstall or partition off someplace and run a fresh install somewhere else to see what it would do... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
nsoveiko n00b
Joined: 04 Dec 2004 Posts: 53 Location: somewhere over the rainbow
|
Posted: Fri Aug 31, 2012 9:12 pm Post subject: |
|
|
nsoveiko wrote: | the only thing this has accomplished is that X11 is now segfaulting as well. |
this is really, really, really weird...
X11 was segfaulting in /usr/lib/xorg/modules/extensions/libglx.so
just for stitzngiggles i reemerged x11-drivers/nvidia-drivers. guess what -- all the problems are gone (knock on wood). |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9675 Location: almost Mile High in the USA
|
Posted: Fri Aug 31, 2012 10:36 pm Post subject: |
|
|
Need a special flag for threads...
* [non sequitur]
Oh well, glad it's working again even though it makes no sense... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|