logs archiveIRC Archive / Oftc / #tor / 2010 / April / 20 / 1
BarkerJr
[warn] Cannot seed RNG -- no entropy source found.
how bad is that?
[warn] Failing because we have 32650 connections already. Please raise your ulimit -n.
I'm thinking those two might be related
katmagic
Why would you not have an entropy source?
BarkerJr
I'm thinking it may not have been able to allocate an fd for it
Faustus
In Gentoo, the default fd limit for Tor is 30000
Sebastian
BarkerJr: that cannot seed rng warning happens because of too many file descriptors. I think it is pretty bad.
Faustus
Maybe in other installs as well
BarkerJr
I wonder why I hit 32650 fd with just 20mbit
actually, it's just 8mbit
Sebastian
BarkerJr: yeah, we're seeing massive trouble on relays all across the board
I wonder if Tor has hit a scalability limit
BarkerJr
one node is using 31746 connections at 8mbit, other is 7494 connections at 14mbit
         

Sebastian
is one of them a guard and the other isn't?
BarkerJr
nope
8mbit = fast, dir, hsdir, 31746 conn
14mbit = fast, hsdir, stable, 7494 conn
also, A8A63DE7C4875FA96DD5A2FF9703E427F67393A9 seems to have dropped out of the concensus but is still burning 2mbit
Sebastian
yeah
my node dropped out, too
arma
fluxe3?
Sebastian
yeah
I changed its port, and it was in the consensus once, I think
since then no more
dr|z3d
Sebastian: Scalability issues you say? Nah, I've been reliably informed scaling is a non-issue.
(Action) winks.
arma
(Action) extends his hack further. current theory is that moria1 is refusing to fetch new descriptors because they would be Unnamed
(Action) increases m&m intake rate to keep up with directory infrastructure intricacies
dr|z3d
(Action) chuckles.
Maybe decrease the beer intake counterwise, arma. :)
arma
(Action) closes in on a bug
nsa
or: phobos committed revision 22208 (/website/trunk/include): comically, osx expert bundles are all that work with the new tor.
or: phobos committed revision 22209 (/website/trunk/vidalia/en): update vidalia 0.2.8, and only osx packages are available.
SwissTorExit
night guys
keb
dr|z3d so we dont need the curly braces around the country codes anymore?
nsa
or: [tor/master] 2010-04-19 22:35:57 Roger Dingledine <arma@torproject.org>: fix "Got a certificate for ?? that we already have"
arma
barkerjr: there
BarkerJr
ya
I restarted my node
Sebastian
arma: maybe we should log its digest or something?
arma
sebastian: works for me. it's a bit messy, in that there are a bunch of digests to log.
do feel free, if it doesn't expand the code too much
Sebastian
How do I reproduce so I can test a potential patch?
arma
Apr 19 19:01:01.179 [notice] We're missing a certificate from authority with signing key 42CF43879B4F1188CB428564EE1D319CE7C502B9: launching request.
moria1 keeps spitting that out
try deleting your 42CF... cert?
Sebastian
haha
pattern not found
         

arma
i think the bug is mostly with logging stuff, not with missing critical certs. so i wouldn't worry too much.
Sebastian
I'm not worrying
but I'd like to give useful output
Because next time I will see that I will wonder "wtf? new or old? What's going on?"
BarkerJr
I get that log line from time on all my nodes
arma
in 40 minutes i'm going to find out if moria1 votes for a few hundred more relays
if it does, i'll commit my hack
Sebastian
Is it in a branch yet?
40 minutes to find a potential bug sounds reasonable.
arma
a branch? no, it's just in moria1's git
Sebastian
is there a diff? still 30 minutes left :)
arma
http://freehaven.net/~arma/fetch-from-vote.diff
BarkerJr
so I'm thinking that tor should stop short of running out of fd and forsaking entropy
Sebastian
yeah. I thought that this should probably be a fatal error
BarkerJr
can we make tor stop accepting connections at like fd-2?
Sebastian
arma: your patch doesn't compile with gcc-warnings :)
(fixing)
arma
your tor still has some entropy. it just didn't add new entropy.
not the end of the world.
Sebastian
arma: either don't comment out that //log_debug, or remove the address parameter, or do this:
- log_info(LD_NET, "Got a non-version-1 cipher list from %s. It is: '%s'",
- address, s);
+ (void)address;
+// log_debug(LD_NET, "Got a non-version-1 cipher list from %s. It is: '%s'",
+// address, s);
BarkerJr
k :)
arma
ok
and the answer is "no, it didn't vote about any more"
how sad.
Sebastian
:/
did it not vote on some of them because it didn't have a descriptor, but others had one and voted on it?
arma
sebastian: i dunno. i'm having trouble getting an example relay that i should be investigating.
Sebastian
ok
arma
plus my tor is 1.2gigs, and my disk can't keep up with my bwauthority
rachel just got a tor weather report about agent being down
it's not down. but it has only 3 votes out of 7.
what a fu*king mess.
Sebastian
yeah. Lots and lots of relays aren't in the consensus reliably the past few days.
arma
it sure would be useful to have a web interface to learn about what the votes say.
Sebastian
:/
arma
v3-status-votes is a horrible interface
Sebastian
indeed it is.
arma
(Action) kills his bwauthority in order to have a computer that works
glurk. bug in my next iteration of the hack.
this whole "poke at it, wait until the top of the hour, repeat" thing is not exactly efficient
but i wouldn't be finding these bugs on a test network i think
keb
hmm my node stopped running today at 13h43 but there is no log entry
ooh a core file
is that any use to anyone?
arma
yes, maybe. what tor version? anything in the logs? what OS?
keb
0.2.2.11-alpha Linux 2.6.25-2-686 #1 SMP
hmm it is more than 400MB file. a couple other log files entris stop at the same time
there are some stats files that were updated at the time it crashed
exact time
i could file a bug and attach the smaller files
arma
don't share your core. it has your private key in it.
deb?
keb
yes
arma
the first step is to wonder if your tor binary has symbols in it
apt-get install tor-dbg or whatever it's called
keb
i built it from source without debug on
arma
building from source should be fine, as long as you didn't strip it
gdb /path/to/tor /path/to/core/
er, without the / at the end
keb
Program terminated with signal 6, Aborted.
#0 0xb7c79947 in raise () from /lib/tls/libc.so.6
arma
pastebin it
keb
ok
http://pastebin.ca/1869019
arma
ah. you have given me exactly not what i wanted :)
type 'where' and pastebin that
keb
updated
arma
updated? i don't see an update
keb
the same url
its at the end of the previous output
sorry i will pastebin again if you cant see it
arma
can't see it. i think pastebin gives you a new number when you change it?
keb
ack yep. http://pastebin.ca/1869022
arma
yes sir ee, that's a bug.
(Action) tries to figure out when it was added
keb
shall i start filing a report
arma
ed174245 (Nick Mathewson 2008-06-18 04:34:52 +0000 953)
hm
that's been around a long time, if that's our bug
i wonder why now.
keb
i just upgraded to libevent 1.4.13 before compiling this tor
arma
any other c developers here to sanity-check me? this code seems clearly wrong.
http://paste.debian.net/69827/
(Action) will time out in a bit and open a flyspray
man, this looks dreadful. like we've been misreporting country stats for karsten, too :}
what is this piece of crap software called tor
keb
i saved the buffer-stats and dirreq-stats files
heh
arma
can you pastebin the last stanza from the dirreq-stats file?
keb
maybe there is a pattern to the misreporting that can be decoded
http://paste.debian.net/69828/
arma
the dirreq-v3-reqs and dirreq-v2-reqs lines are the one with the bug
hm. looks like the output we expect. why the heck does this work at all. maybe i'm just being blind and it's not a bug.
(i have been hacking moria's directory stuff for the past many hours. brain not so good anyway.)
keb
heres one that includes the last dirreq-v3-reqs and dirreq-v2-reqs until the end http://paste.debian.net/69829/
arma
oh. ha. i know how it's working.
it should not possibly be working, but it is. through luck.
we are allocating 32 bytes on the stack, writing something like md=8 into them,
adding them to a smartlist, unallocating them, repeating.
at the end, we have a smartlist of a bunch of unallocated 32 byte entries
we join them all up together (reading the unallocated memory, which is bad),
then we try to free each of the unallocated addresses, which are on the stack
keb
yikes
arma
that last step is where your tor decided it had had enough.
keb
how did it work for so long
arma
quite so.
keb
ooh the previous Tor instance stopped exactly 24h before that. so this was the first stats output
arma
i bet all our tor relays are crashing if they stay up long enough to write out a dir stats line.
keb
(i stopped it to upgrade from 0.2.2.10 to 0.2.2.11
arma
hey, this dir stats line thing is still optional, right?
keb
yes
arma
that would explain why not many people have run across it.
keb
so not many people are contributing to stats :|
arma
i remember fondly the days when i reviewed all or-cvs commits :)
keb
now we leave it up to the nsa to sanity-check them
or is that an irc bot ;)
arma
DirReqStatistics is what it's called?
keb
i had these options: CellStatistics 1 DirReqStatistics 1 EntryStatistics 1 ExitPortStatistics 1 ExtraInfoStatistics 1
xtoaster
wow, what do you need these stat for ?
keb
http://archives.seul.org/or/dev/Jul-2009/msg00002.html
oops i am suposed to be sending these to Karsten :/
i remember sending the package once anyway
arma
keb: no need to send them to karsten anymore. they get put into your extra-info descriptor automatically.
xtoaster: some of them are how we generate graphs like http://metrics.torproject.org/recurring-users-graphs.html
keb
ah ok i see that in the next-by-thread
arma
xtoaster: others are how we are figuring out why tor is slow
keb: https://bugs.torproject.org/flyspray/index.php?do=details&id=1365
(thanks!)
xtoaster
oh i see.
keb
cheers
so nobody copy/pasted that code to the other stats reports
arma
hope not. copy-paste is a sign of bad programming.
xtoaster
:-D
keb
lol oh well Apr 20 00:50:32.778 [warn] Failed to parse/validate config: While Tor is running, changing either CellStatistics, DirReqStatistics, EntryStatistics, or ExitPortStatistics is not allowed.
arma
keb: ah ha. the answer is, the bug didn't happen in 0.2.1.2-alpha, it happened much more recently
Date: Sun Feb 28 21:46:50 2010 -0500
that's when nick broke it
which didn't get merged into until apr 2
keb
so bad code was there but not used?
arma
meaning the bug is new in 0.2.2.11-alpha
no, the code was good before. it became bad just for 0.2.2.11-alpha.
nsa
or: [tor/master] 2010-04-20 05:09:40 Nick Mathewson <nickm@torproject.org>: Switch geoip_get_request_history to asprintf; fix bug 1365
arma
looks like my directory hacks are working
nsa
or: pootle committed revision 22210 (/translation/trunk/projects/torbutton/es): updated files from pootle
or: pootle committed revision 22211 (/translation/trunk/projects/website): updated files from pootle
or: [tor/master] 2010-04-20 06:48:35 Roger Dingledine <arma@torproject.org>: minor cleanups
« prev 1 2 3 next »