logs archiveIRC Archive / Freenode / #exim / 2015 / August / 17 / 1
hds
hello
do you know why db/retry is keeping its size after exim_tidydb?
I executed this exim_tidydb -t 0d /var/spool/exim/ retry
but retry file is keeping its size
and it is huge
1GB aprox
rjek
I would like a way of removing cache items from exim's databases, specifically the call out cache, by domain.
I often have problems where Exim has cached that an address does not exist and then somebody goes and creates it :)
Any ideas beyond deleting the whole callout cache?
henk
set callout_positive_expire and similar options differently
rjek
henk: That would reduce the problem but not solve it?
And mean when there haven't been changes made more load
henk
the only way to completely "solve" the problem would be not to cache anything AFAIU &
rjek
What I want is a way to remove items from the cache related to a specified domain; I can trigger that on configuration change.
ie, user creates a new alias/mail box in a domain, during pushing of new config to front ends remove related cache entries
henk
because 2 hours is way too long but only for that one domain?
rjek
Yes.
ie, if somebody has made a configuration change to a domain, that cache data is now completely out of date and I'd like to remove it
henk
why is 2 hours too long? and why do you not simply delete the whole callout cache?
         

rjek
2 hours of us refusing mail because it thinks the mail box doesn't exist when it does is a pain.
And I don't want to delete the whole cache because there's still useful data in it
I just want to remove the data I know is wrong
henk
AFAIK there is no such solution as you ask for and IMHO you are making it way too complicated without (telling me) a good reason &
rjek
2 hours is too long to wrongly 500 mail. The cache contains good acurate data I don't wish to destroy.
henk
you can use exim_fixdb &
and how do you get 500 legitimate mails in the first two hours of the life of an email address?
that is not normal.
rjek
Somebody tries to send an email to foo@bar.com, it fails because it doesn't exist, realise they need to create the alias, create alias, Exim still refuses mail for 2 hours...
I get this exact situation from my users all the time :(
I couldn't work out how to use exim_fixdb to do this
henk
You are not really answering my questions, merely repeating them is not an answer & good luck
rjek
Then I'm not understanding your questions, sorry
The problem is that users are awful and end up trying to use email addresses before they've created them, meaning Exim caches the non-existance of them
henk
IMHO you are making a problem out of this where there is none & 2 hours is not a long time IMHO and its usually only a problem for 2 hours if someone does something stupid & thats life, live with it. Also you have the possibility to reduce it to 0 hours, but for some reason you dont really explain you dont want to do that.
rjek
Reducing the cache lifetime to 0 hours negates the usefulness of the cache.
2 hours is a really long time for angry users.
I simply want to tell Exim that its cache data for a given domain is out of date immediately when such a configuration change happens
(I imagine the inverse problem exists too; when a user removes a local part)
henk
No, it does not, because positive results are still cached.
2 hours is 2 hours and that is not a long time in the context of email &
rjek
So it'll have cached that the address is valid, even though it no longer is...
henk
What configuration change? Why is there a configuration change?
rjek
henk: Perhaps for some people
henk
Yes o_O you seem to be confused about the concept of "caching" &
rjek
Users are free to create aliases and mailboxes on the backend server as they see fit; front end servers query it
I'd like those changes to cause cache data removal from the front ends.
henk
If you want accuracy: dont cache.
rjek
Because otherwise it means the front ends will sometimes incorrectly reject the mail
henk
its as simple as that, but you dont seem to want to accept that &
rjek
Removing stuff from caches when you know the data is out of date is hardly unusual.
henk
IMHO it is &
in what context is it usual to do that? Ive barely ever heard anyone do that &
         

rjek
Cache item removal is not limit to time. It can be space, or replacement data.
This situation is replacement, more up-to-date, data.
It would be unfortuate if your CPU's cache was only time-based, or if Squid's cache couldn't cope with items being updated.
Simon-
rjek: why is it even entering the callout cache?
rjek
Simon-: Because people are awful :)
Simon-: Twice this week I've had users give an email address out that doesn't exist yet, realise that when they get reports of bounces, create the alias/mailbox, and then Exim's cached the fact it doesn't exist and continues rejecting mail to it until the cache entry expires
Simon-
so don't use verify for local domains?
rjek
The front end servers have no local domains at all
They're all remote.
Simon-
if you have special incoming only servers you can reduce the global cache timeout
rjek
Yeah, that mitigates the problem but doesn't solve it :)
Simon-
then have incoming email go directly to servers that know which users exist
rjek
Sadly not an option for us :-/
Simon-
if you want to remove a specific entry you could modify the code that clears old entries
rjek
If we didn't have 15 years of technical debt everything would be jolly.
Simon-: I have considered writing a problem that locks and modifies the DB; it's a sleepcat-style one I think?
But I was wondering if there was an official way of doing this
Simon-
not that I'm aware of
henk
Seems you dont like any option thats possible, so AFAICT all thats left as a possibility is modifying exims code to do what you like. I still have no idea what that is, though. Probably make exim_tidydb domain aware or something &
Maybe split up spool directories by recipient domain or something like that might help & Not sure if exim can already do that or if that would require code modification as well.
rjek
Seems like the options that are possible don't solve the problem...
I think looking into modifying exim_tidydb is probably the best option here.
henk
Sure, reducing negative caching to 0 would definitely solve the problem, but you dont like that either &
rjek
Because that negates the whole point of having the cache, there's nothing to like about that solution.
henk
If you dont like the cache, dont use it and provide a complete dataset to the frontend servers &
Because in the end thats basically what you want &
rjek
No, what I want is to be able to tell Exim when its cache data is out of date.
henk
how do you propose that should happen?
rjek
I have no time to repaste the above discussion.
But it looks like patching exim_tidydb is an option, so I'll work on that.
Simon-: Thanks for the pointer!
henk
uhm, right, why say thanks to henk at all & its not like he said anything useful whatsoever o_O
hs12
rjek: I didn't follow all details of the above discussion, but why can't you use exim_fixdb for cleaning the entries you do not want there in the callout cache?
echo -e 's.schenk@example.de/<technik@foo.bar.de>\nd' | exim_fixdb $(exim -n -bP spool_directory) callout
rjek
hs12: For simplicity, I was hoping to do it for any entry for a given domain
But I will look into that, thanks
hs12
exim_dumpdb <spool_dir> callout | sed '...' | exim_fixdb ...
Though I'm not sure if exim_dumpdb and exim_fixdb want to run at the same time. Maybe you need to create a tmp file first.
rjek
Looks promising!
bjornar_
Anyone with a clue how to use the spf lookup (_lookup_)
trying ${lookup{gmail.com}spf{127.0.0.1}} but get pass
...guess softfail should be correct here..
jgh_
I don't see any documentation for that. Where are you looking?
bjornar_
code
jgh_
what code?
bjornar_
seems I got tricked by myself.. 127.0.0.1 is treated internaly seems
in libspf2 .. to allways give pass!
what code!?
1 2 next »