logs archiveIRC Archive / Oftc / #tor / 2010 / June / 29 / 1
katmagic
http://l6nvqsqivhrunqvs.onion/index.php?do=topic&id=11392#post53706
nickm
It's hard to tell an attack from an experiment-gone-wrong. But I'm pretty sure that all of the nodes in question got booted by the directory authority ops within a few hours of getting added.
atagar
nickm: weren't they up for roughly three days before being noticed?
nickm
were they? Whoops.
katmagic
Yep.
nickm
Wish somebody had said something to the authority ops sooner.
The network metrics server should really include "email the authority ops when a bunch of nodes show up too fast" code.
katmagic
Actually, they were noticed on Saturday. http://l6nvqsqivhrunqvs.onion/index.php?do=topic&id=11392
atagar
nickm: it'd be nice if he had automated scans, even just a basic alarm to say 'Oy! We just got a ton of new relays! Maybe this is good, but someone should check'
nickm
he who?
         

katmagic
Apparently I'm the only OnionForum reader who comes in here.
atagar
ack, s/he/we
nickm
atagar: Did I just hear you volunteer to write such code? :)
atagar
... hmm... ok
nickm
woot
katmagic: it could well be.
atagar
nickm: I'll write up a quick little script tomorrow, but if we're gonna be running soat consistently with alarms in irc then that would be the appropriate place for it
nickm
atagar: IIUC, SoaT is for detecting individual nodes behaving badly, not for detecting worrisome node trends
atagar
true, or we could view it as the automated health check of the tor network and hence something to be expanded with these sorts of metrics
... but then again, 'do one thing and do it well'... maybe not such a great idea
nickm
yeah
atagar
oh well, in that case we should banter the idea of a new project and what sorts of metrics (other than relay count) might be of interest
nickm
Well, there's already the network metrics code that karsten et al have been working on.
for all I know it's already testing for this and sending email alerts someplace nobody has been looking. :(
atagar
karsten: thoughts on having alarms for weird events (ex, sudden changes in user / relay count)?
(I wonder how long it took for us to realize CN was really good at blocking bridges, too...)
nickm
That is one of the explicit goals of the improved metrics code, IIUC.
but when karsten's around, he can explain better, and probably point out places where you could help
s/you/people/
s/people/people, including you,
atagar
metrics is all java and python, so happy to help if he wants it
nickm
otoh, if it's not currently ready to do this, it'd probably be worthwhile to deploy a cron job today in case adding it to metrics takes longer than expected
atagar
agreed
karsten
atagar: you can extend ERNIE to add such sanity checks, if you want to.
nickm
karsten: does ernie currently do email alerts? And could it? And will you be annoyed if I don't write it in all caps?
karsten
heh. not yet. if you know how to do it in java. nah. :)
it might make sense to do this in ERNIE rather than writing yet another script for it. ERNIE takes care of downloading the most recent consensus (and other docs if you need them).
nickm
respectively: That's what I thought. I think atagar is interested; sending email in Java is supposed to be easy. That's good; I grew up being told that having your name written in all-caps was a sign of excessive self-importance. ;)
         

karsten
you would add another data sink that takes the consensus or previously parsed parts of it, uses some temp file if you need to, and sends out the email notification you want.
not at all. ERNIE is just a little bit proud it's a real acronym, nothing else. :)
it's also yet another working name that waits for a real name to come by.
atagar
unless there's advantages to using ernie I think I'll just hack up a little python script for relay count
karsten
atagar: well, as i said, the advantage of using ERNIE is that you don't have to worry about getting the relays. they are downloaded anyway every hour.
up to you, i guess.
atagar
isn't that the case for relays anyway?
karsten
erm, when i said relays i meant descriptors.
what do you mean?
atagar
with FetchUselessDescriptors just attaching TorCtl to a relay should yield the same results, yes?
karsten
unsure if that gets you the most recent consensus.
atagar
oh, if only we had a way of manually refreshing descriptors periodically (hint, hint nickm)
reguardless, the network status is all we need for counts, yes?
karsten
yep.
atagar
spiffy
karsten
if you want uptime or platform, you need the descriptors.
katmagic
Doesn't HUP refresh descriptors?
karsten
atagar: out of curiosity, why do you prefer starting from scratch here?
atagar
(a) it'd be far faster, (b) sounds like ernie is already for graphing so it seems kinda hacky to introduce alert/email functionality, (c) the "R" in its name inspires terror that I won't understand half the codebase, (d) using TorCtl is both lighter weight and I can easily get it up and running on my own relay
of those, a and d are what I care about most personally
karsten
a) may be true for the moment, but it might be someone else wants to extend your alert code, and then it makes sense to not have n projects for n things.
atagar
karsten: heh, I just had this discussion with nickm where I argued that it should be in soat :P
nickm
It _is_ a better match for Ernie, though.
atagar: have you looked at the ernie code at all?
karsten
b) is not entirely true. ernie processes data. preparing data for graphing is just one thing. it also collects descriptors for the tarballs/exonerator, sanitizes bridge descriptors, and generates the consensus-health page that is more like a network status thing.
c) no need to use R here at all. just java.
katmagic
Eek.
karsten
d) this would be a useful thing to run on some tpo server, not your relay.
if you want to look more at ernie, look at src/ConsensusHealthChecker.java and how it's called from src/Main.java.
atagar
nickm: Disagree that it's a better match for ernie. Ideally soat will be constantly running, giving alerts when bad things happen in the tor network (just specialized to be bad exits). Including more general alarms like 'relay count stairstep function' makes a lot of sense to me.
karsten
if you can integrate this in soat, great. that's much better than starting a new project, imho.
atagar
I agree that this is small enough that it should be part of either ernie or soat.
Though I think it makes nickm twitch :P
karsten
ok. i should continue with my original plan to go to bed. if i can help you with the integration of your shiny new java app that parses consensuses and decides when to mail someone, just let me know.
it may not be trivial to decide when to send out an alert. the tor network grows and shrinks over the day. and maybe you want to discover trends over a week, too.
there may be even more anomalies to check for.
arma
i could see an argument for having it be part of soat
lots of ways to be a snake
karsten
i think either ernie or soat are fine. just not project 113.
and now off to work on my original plan..
katmagic
Project 113?
atagar
(Action) shamefully knows he's favoring soat quite a bit since it's python...
katmagic: he just means we should stop spawning new projects for every bit of functionality
boxbeatsy
hi, is there anything different about setting up Tor on an EC2 instance?
arma
shouldn't be
boxbeatsy
word, there isn't an AMI available with Tor already set up right?
honestly i shouldnt be asking questions until i've tried it out, but i need to do this quickly
arma
not that i know of
boxbeatsy
ok thanks
arma
what's the hurry?
boxbeatsy
have a deadline to meet
arma
a deadline for setting up a tor relay?
boxbeatsy
lol need tor to get a bunch of data that i need to write my term paper
arma
more details?
boxbeatsy
so i'm basically modelling google's search ranking algo, and i need to scrape a bunch of data from the SERPs and each ranked site
i have a huge budget on EC2 so I'm setting up Tor to run all my scripts
arma: i know tor is slow, is the latency for a GET request to a SERP more than a few seconds?
on avg
arma
what is a SERP?
boxbeatsy
sorry
search engine result page
arma
are you trying to use tor to scrape google? they don't like that, you know.
i hope you're ready to solve a lot of captchas.
boxbeatsy
yea i know, but i'm not being evil ;)
ill be fine with the captchas
arma
yes, you are. you're making it so other tor users are forced to solve a captcha if they want to do a google search.
boxbeatsy
nah, actually google doesnt captcha on SERPs
unless you're doing advanced queries
arma
ah. so one query, and then you pull down all the answers it provides?
boxbeatsy
yea, most of the operatin time is in actually hitting each site that is returend to analyze its on page optimiziation
arma
ok. less evil.
still, why tor?
boxbeatsy
some of the API's i'm using rate limit/IP
ok ima get to it, thanks for yoru thoughts arma
arma
(Action) goes to wash off the seo smell
boxbeatsy
hi guys, is it possible to open multiple tor connections on ubuntu?
sahal
what do you mean?
boxbeatsy
well, i am using TOR to run multiple instances of a script
and i want to know if each oen can have it's own TOR connection, and therefore have unique IP's
*one
sahal
you could specify a seperate exit node for each instance of the script
boxbeatsy
ah ok thanks
sahal: how exactly do i do that
i should be searching more first, but im in a hurry
sahal
http://en.wikipedia.org/wiki/.exit#.exit
katmagic
That's deprecated and insecure.
It won't work without putting "AllowDotExit 1" in your torrc.
sahal
oh oops
boxbeatsy
is there another way to open multiple TOR connections then?
katmagic
What exactly do you mean by multiple Tor connections?
Do you want several programs to be able to use Tor?
boxbeatsy
yea exactly, but with different IP's
sahal
(s)he's running some kind of scrape/attack/brute force script that needs to look like its coming from different places
katmagic
Tor will normally rotate the exit periodically.
boxbeatsy
lol it's a scraper....so the lesser of those evils
ok but i cant use more than one exit node at once?
katmagic
Yes, you can.
If you issue a NEWNYM signal, your old connections will use different circuits (and hence exits) than your new ones.
boxbeatsy
gotcha, thanks!
Sebastian
note that Tor ratelimits newnym requests. We don't like people using Tor who are just after getting many different IP addresses, you're putting a big load on the network.
boxbeatsy
hi, how do you set your tor ctrl password?
dr|z3d
boxbeatsy: Personally I use Vidalia.
boxbeatsy
i need to interface it programatically via telnet
but i cant figure out how to change my auth_code
dr|z3d
HashedControlPassword might prove a useful directive.
(man should have further details.)
boxbeatsy
dr|z3d: gotcha, thanks much
carnivore
Any news on who opend all those nodes yesterday?
arma
no news. i mailed a friend of mine at princeton, but i think the evidence is disappearing too quickly for them to be able to figure it out once they read my mail.
katmagic
So I guess this means it was malicious, then?
arma
well, i'm always the optimist about these things, but there's no additional reason to think it was malicious
somebody screwed up, noticed, stopped screwing up. does that mean it's malicious?
ilter_
Hello. I'm looking debug logs of my relays and i realized that onionskin_answer() method is called so more than circuit_deliver_create_cell(). One of them is called 2334 times and the other one 120 times. I couldn't understand how it can be possible. Could you help me to comment this situation?
As far as i understood onionskin_answer() method is using to send "CREATED" cell to previous hop, and circuit_deliver_create_cell() is using to send "CREATE" cell to next hop. So i think that their calling times should be same. Could you say where is my mistake?
fear
Why Ticket #1291 diagnosed depends of Running from Valid? dirserv_set_router_is_running() works without Valid flag. What a hidden negative feedback turns off Running flag else?
Sebastian
fear: hm, good question.
router_is_active depends on the Valid flag, but I can't see how that would influence Running.
fear
well planetlab2 is non-Valid and Running right now. So ticket is wrong or what?
Sebastian
I think it is wrong.
But there is some kind of influence that invalid relays lose their running flag after some time
Except I'm not seeing why this would happen
MeCooL
(Action) Hi
atagar
as a follow up for yesterday's discussion (in case others were tracking it...) here's a little script that provides email alerts when the number of relays change sharply: http://tor.pastebin.com/UGuay2ah
MFen
can anyone help me get tor+firefox going? i have polipo running. i have enabled socksParentProxy=localhost:9050 and socksProxyType=socks5 in polipo, and restarted it. tor is running. netstat shows 9050 and 8123 listening. foxyproxy has a tor proxy, set to localhost 8123, socks proxy, socks v5, and that proxy is selected.
one thing that is confusing me is that i can connect to localhost:8123 via http (in my browser) and get a web page/configuration interface. shouldn't that be speaking socks?
nm, i got it now. the proxy setting in firefox should NOT be socks.
« prev next »