logs archiveIRC Archive / Oftc / #tor / 2010 / January / 18 / 1
mete888
is it possible that tor has a memory bug?
my tor is blowing up the last hours...
j_r
memory leak
possibly
mete888
cpu: 4.7% mem: 113 MB (5.6%)
that shows arm
every hour the mem get 0.5% bigger
arma
https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#RelayMemory
a lot of the reason why tor grows in size is because glibc doesn't give back any memory that tor frees.
if you build with the openbsd malloc, it will suddenly look a lot better.
nsa
or: mikeperry committed revision 21424 (/torflow/trunk/NetworkScanners):
or: Add more verbose breakdown on flags. Look at how much
or: underloaded 0-10% guards are vs 10-20, 20-30, and 30-40. They
or: keep getting more overloaded (still! - or rather, again!)
mete888
hmmh, bad
nsa
or: Christopher Davis <chrisd@mangrin.org>: 2010-01-17 23:14:51 [polipo/master]: Merge branch 'fleischer-dc17'
or: Christopher Davis <chrisd@mangrin.org>: 2010-01-17 22:46:21 [polipo/master]: Optionally avoid transmitting sensitive information about a user's system.
or: Christopher Davis <chrisd@mangrin.org>: 2010-01-17 22:55:32 [polipo/master]: Don't add port to Warning: headers.
polipo: chrisd@torproject.org committed patch at Sun, 17 Jan 2010 23:17:11 +0000 (UTC) to polipo-commits: [polipo/master 1/2] Optionally avoid transmitting sensitive information about a user's system.
polipo: chrisd@torproject.org committed patch at Sun, 17 Jan 2010 23:17:11 +0000 (UTC) to polipo-commits: [polipo/master 2/2] Don't add port to Warning: headers.
arma
weasel: is there an echo in here?
weasel
arma: ?
         

arma
is nsa saying the correct amount of lines for the two commits?
weasel
does it not?
arma
well, do we want it reporting to 'or' and to 'polipo' at once?
Ganneff
i would say its the same as last time - a doubled commit script running and nsa just does what it has to - forward the mails to irc
weasel
ah.
see, being crypting isn't helpful.
arma
ah. i figured 5 lines for 2 mails was self-evidently wrong
weasel
no.
it's not self evidently wrong.
arma
anyway. yeah, what ganneff said.
weasel
moving mail from polipo-git to /dev/null now
that should help
arma
ok. what is the other one? (if there's a duplicate, and polipo-git was the one you removed, which one remains?)
(i fear we are still failing to communicate. i did not mean "please get rid of all polipo notifications to #tor".)
weasel
the one where the git post commit hook mails nsa directly instead of relying on my score of scripts to magically parse the commit message out of a mail
arma
ok
great
nsa
or: mikeperry committed revision 21425 (/torflow/trunk/NetworkScanners):
or: Make output slightly more sane.
or: mikeperry committed revision 21426 (/torflow/trunk/NetworkScanners):
or: Add a key.
or: Roger Dingledine <arma@torproject.org>: 2010-01-18 00:41:22 [tor/maint-0.2.1]: stop bridge authorities from leaking their bridge list
arma
wonder how many other problems like that are lurking. i found that one by having extra time on a train and just looking through the code.
mikeperry
wow
was it possible to just request the entire list of descriptors from them if they had a v2dir port or something like that?
arma
mikeperry: worse than that. you could fetch the router stability file.
so, all the identity keys they were tracking, and what they thought about them
http://216.224.124.114:9030/tor/dbg-stability.txt
mikeperry
is that fix already in master?
I only saw a commit for maint
arma
nope. i should merge it into master sometime.
phobos
...why not now?
arma
going to put this bug 1217 patch into 0.2.1 too
anybody have a link for nickm's openssl patch that reduces buffer memory?
         

nsa
or: phobos committed revision 21427 (/projects/misc-sysadmin): update server names and memory allocation.
Lucky
Tonga and Lifuka's RAM has been increased from 2GB to 3GB, which should help performance a litte. Improvements on the networking side of things will have to wait until a couple of weeks from now. (Lifuka remains offline networking debugging. Tonga is back in operation).
arma
great
Lucky
Likely the biggest performance increase would come from my upstream installing the two new switches that sit unused in their boxes...
Until then, I'll have to tweak around the edges.
arma
you might like our patched openssl too, then (see the or-talk list)
ioerror_kef
Lucky: I think that's probably true
j_r
sjmurdoch: ping
hrrm
I guess its 5am
Lucky
ioerror_kef: yeah, that switch installation seems to be hampered by lack of manpower. I should take a trip to the colo and volunteer my time.
ioerror_kef
Lucky: I know a box that could use some love ;-)
Lucky
ioerror_kef: not inconceivable that I might be there in the next 3 months.
Been away from there for too long.
Uh. Guess that's nottor.
arma
ok. well. it is time to beat the crap out of the current fast Guard nodes.
i wonder how they will like having 50000 tls connections open.
or is that 100000.
Lucky
arma: are you running tests against the live network?
arma
we're rebalancing the network again, after mikeperry found a bug in how tor clients select their first hops
you might enjoy lurking on #tor-dev
see also https://bugs.torproject.org/flyspray/index.php?do=details&id=1217
rieo
so flags = CRN_NEED_GUARD from choose_good_entry_server() for user's config UseEntryGuards 0 too? with it no matter if state->need_uptime == 0, routerlist_sl_choose_by_bandwidth() calls anyway.
arma
yeah, i had been wondering that. i think the answer is yes, we should send NEED_GUARD in even if useentryguards is 0.
we want users to weight their hop selections based on the same way other users do. 'useentryguards' then decides whether you stick with the entry guards you picked, or drop them after using them.
in particular, there aren't many reasons to set useentryguards to 0. our torperf runs are one good reason to do it -- when you want to sample the whole network.
rieo
before flags it works so: "state ? 0 : 1"
arma
yep. my patch changes the behavior.
(both the current behavior and the previous behavior)
well, actually
ah. hm, no i was confused. ignore my 'well, actually' :)
rieo
I guess actually state->need_uptime == 1 anyway forever.
arma
hm? i hope not. it should only be 1 for circuits built with need_uptime in mind. it's designed for streams with destination ports in LongLivedPorts.
rieo
ah need_capacity, wrongly types. sorry
arma
and for introduction point circuits, for hidden services
i think introduction points set need_uptime to 1 but need_capacity is still 0
they're one of the few exceptions
rieo
ok, I talked about need_capacity.
arma
need_capacity will usually be 1. but not always.
rieo
so why entry need weighting always?
arma
a fair point. it doesn't. that's just how nickm did it in the fix for bug 440, and it's close enough to right.
entry needs weighting always because you'll reuse the guard you pick for other circuits, most of which will want you to have picked it weighted by bandwidth.
the only exception i guess might be if useentryguards is 0, in which case you won't be reusing it.
the question comes down to what people who set useentryguards 0 expect. we keep finding these edge cases where we're not sure what behavior we should provide for an option that ordinary people shouldn't touch.
for torperf, which is the only actual use case i know of, we want it to imitate what a real client would do.
and torperf doesn't run a hidden service, so it doesn't matter there
rieo
ok.
Sebastian
I've been thinking about making a new special-purpose config option section in the manpage
So that regular Tor users/hidden service providers can read the top of the manpage, and know every option that is relevant
arma
good idea
Sebastian
Let's do that once we have asciidoc.
arma
how's that coming?
Sebastian
Autoconf still needs some beating
arma
we could just run it manually for now
Sebastian
It already works if autoconf is installed
it doesn't work if it isn't installed
also, I wanted to try and see how the make dist deals with it
but then I totally failed to make a tarball, because of tons of madness
one of them being that the tor source needs the website, which needs the tor source to build
-.-
but what we'll want is to fix making the website
except I cannot make it on os x
arma
make dist needs the website, which needs make. not make dist.
so it isn't quite circular. :)
Sebastian
I know it's not circular
but to build the website, we need the manpage in Tor
So I have to patch the website build first
this is all pretty crazy
The website build process needs Tor and Tor-stable only to build tor-manual and tor-manual-dev, right?
arma
right. you could just comment those lines out, in tor-manual.wml and tor-manual-dev.wml
Sebastian
Also, I wonder whether we really want the website in the Tor source tarball.
arma
or delete them
Sebastian
Including the translations
arma
a good question. we want *something* for people who get tor and don't know how to use it and can't reach our website.
if it's not in the tarball, it isn't in the windows bundles.
Sebastian
ah, it is in the windows bundles?
it isn't in the os x bundle
rieo
what about "the best possible selection we could do that, since it's weighted by the bandwidth right now, rather than weighted by the bandwidth numbers at some point in the past up-to-two-months when we picked the node."
choose_random_entry() could be do routerlist_sl_choose_by_bandwidth() always.
arma
sebastian: no idea if it's in the windows bundle. but it could be. somebody needs to figure out what users actually need from our bundles. perhaps the right answer is to improve vidalia's help windows. not just shove a bunch of html files at the user, without even telling them they're there.
Sebastian
right
that's what I mean
also, how would users go to fetch them
they have a vidalia icon, and they click it
arma
sebastian: speaking of which, vidalia's help text is not on track to be translated. it's not in pootle, and we deleted any mention of it from the translation portal.
Sebastian
I suspect many won't go into the (hidden by default) programs folder and look around
that's sad.
hrm
arma
rieo: we can't weight our guard selection process by bandwidth, and *also* weight which of our guards we use by bandwidth. then we don't load balance correctly.
Sebastian
So I think killing the website from the tarball would be a good thing regardless. if we don't do that, we should actually select the websites we want to ship, because hidden services and switchproxy seems useless
arma
rieo: we could choose our three guards uniformly at random, and then generally just use whichever is fastest. but the load balancing isn't very good there either.
rieo
but they can be chosen 2 month ago, many changes.
arma
rieo: yes, i agree.
we're screwed either way. but i think we're less screwed with the current approach.
(well, the proposed new current approach ;)
fabi
hi
arma
hi
fabi
I want the authorities of my private network to exclude a certain set of routers from their consensus. These can be identified by their port. I put a AuthDirReject *:<port> to their torrc, but that doesn't work
i can't find any hint in the logs, that they are trying to reject anything
arma
make some new logs?
fabi
have you ever used this feature and it must be a bad configuration or does it possibly not work at all?
arma
it's possible it doesn't work at all
i don't think anybody intended to use it with ports
i've used authdirreject, but i did it with an ip address
fabi
yes, that' actually what i thougt. iirc it worked a few weeks ago with ip adresses
but i run 2 ORs on one ip adress and only want sometimes to eliminate one of them from the consensus (fast)
arma
list it by identity key with a !reject line in the approved-routers file?
like so:
!reject FFF1 1F6F D6DF BD59 045D 8380 ACDA 91B7 793A 59B8
fabi
hmm.
how long would it take for the authority to recognize it?
arma
you need to hup your authority
fabi
cool
waht i actually wanted to have is sth. like a switch for some of my routers to turn them on and off in consensus
arma
well, feel free. the code's not that hard. :)
fabi
(they behave malicious, and shouldn't be present all the time)
i'll try that approved routers thing. thanks
oha
Jan 18 07:00:19.373 [info] dirserv_add_descriptor(): Did not add descriptor from 'CHEATERentry' (source: 134.76.81.92): Authdir is rejecting routers in this range..
Seems the pattern matches...
Sebastian
ok, so neither tbb nor the install bundle ship the website.
which is a good move, I think
most people cannot open .html.en files anyways
arma
sebastian: so we should just stop putting them in the tarball.
and see who complains
Sebastian
(Action) makes patch.
oh hey
I should go buy groceries
that'll be a fun adventure -.-
fabi
ok, i found sth interesting
if you add a authdirreject *:<port> the authority seems to reject _any_ ip adresses :)
arma
could be. you're in undocumented territory
fabi
hehe, i'll better turn it off. my half network is'nt in consensus anymore ;)
rieo
is it good idea do not mess circ for streams with defined exit with another? at least this can repair a core of many edge cases.
Sebastian
rieo: Can you try to rephrase that? I didn't quite understand what you mean
Do you mean we should never cannibalize certain kinds of circuits?
rieo
never chose (for any case) it such circ was builded for streams with predefined exit (enclaved as example). the same way as tor do not messing one-hop circs with another for regular streams.
Sebastian
Ah! If I understand you correctly now, yes, that's exactly what I suggested before, too.
rieo
it's need new property for circ_t
Sebastian
If we build a circuit that has an exit not chosen by the random exit choosing logic, but rather because it used mapaddress, .exit, enclave, hidden service, do not use that circ for anything else.
is that it?
rieo
yes
Sebastian
Let's see if someone answers here until I'm back, if not I'm making a bugreport.
rieo
needed to change struct is cpath_build_state_t
« prev 1 2 next »