Issue562

Title Gmail (google) marks registration email as spam
Priority bug Status chatting
Superseder Nosy List Tim.Tisdall, cpburnz, fleminra, montanaro, r.david.murray, socketpair, ztane
Assigned To Topics

Created on 2015-01-25.03:44:16 by cpburnz, last changed 2016-07-12.12:54:07 by Tim.Tisdall.

Messages
msg2930 (view) Author: cpburnz Date: 2015-01-25.03:44:15
Gmail (google) marks registration emails as spam. This occurred when registering for both <http://bugs.python.org> and <http://psf.upfronthosting.co.za>. Gmail indicated they were marked as spam because "It is in violation of Google's recommended email sender guidelines.  Learn more <https://support.google.com/mail/answer/81126?hl=en&authuser=2#authentication>".
msg2931 (view) Author: r.david.murray Date: 2015-01-25.17:01:35
Mail is consistently sent from report@bugs.python.org, always from the same IP address, 46.4.197.70.  46.4.197.70 resolves to bugs.python.org.

So, we are meeting the requirements set forth in the article you link.  I've asked about adding SPF records, which I don't *think* we currently publish, but per the linked doc, that shouldn't be *necessary* to satisfy google.

We have a suspicion that they are marked as spam because spammers fake from addresses and send messages to the tracker, which automatically sends back the registration email (since it hasn't seen that email address before), and the from address recipient marks the message as spam...which information then makes its way into one or more pattern recognition databases.  At this point I'm not sure there's much we can do about it, though I would be in favor of eliminating the auto-registration response (ie: generate a registration email only if registration is requested via the web interface, not implicitly via email).  But I'm not at all sure that fixing that now would have any affect on whatever database it is that has the registration emails marked as "likely spam".

If I'm right about the above, though, it means that google's bounce email is...well, lying is probably to strong a term, but misrepresenting, at least...the reason the email is being marked as spam.
msg2947 (view) Author: fleminra Date: 2015-02-06.21:01:46
Gmail is also categorizing non-registration e-mails as spam. Specifically when a new comment is added to a Python bug, Google marks the generated e-mail as spam.

(Doing my part to unmark the spam flag...)
msg2948 (view) Author: fleminra Date: 2015-02-06.21:03:28
In reference to my previous comment: "new comment" e-mails from this tracker (psf.upfronthosting.co.za) are *not* being marked as spam.
msg2963 (view) Author: ztane Date: 2015-05-09.02:42:09
The new comments on the Python bug tracker also go to spam. I investigated the mail further, and there are *several* issues there.

First: "Use the same address in the 'From:' header on every bulk mail you send."
- the mail I received from meta tracker registration had

    From: PSF Meta Tracker <metatracker@psf.upfronthosting.co.za>

but the `Mail From` line and return path had `roundup-admin@psf.upfronthosting.co.za`.

Second: still no SPF record!

Third: "Keep valid reverse DNS records for the IP address(es) from which you send mail, pointing to your domain." there needs to be a *reverse*. Well, the headers show that the mail was `Received: from psf.upfronthosting.co.za ([2a01:4f8:131:2480::3])`, which is an address belonging to Hetzner. Well, guess what, that IP address does have neither forward nor reverse mapping *at all*, that is `host 2a01:4f8:131:2480::3` does not give out PTR record for `psf.upfronthosting.co.za` but neither does the domain have an `AAAA` record:

    psf.upfronthosting.co.za has no AAAA record

At this point it would look to google equal to claiming to be sending mail from `microsoft.com`.
msg2964 (view) Author: ztane Date: 2015-05-09.02:46:48
The above is true for new comments/bugs as well - they end up in spam:

    Received-SPF: none (google.com: roundup-admin@psf.upfronthosting.co.za does not designate permitted sender hosts) client-ip=2a01:4f8:131:2480::3;

Except there is 1 more peculiarity which smells even more wrong:

    From: Antti Haapala <report@bugs.python.org>

Why does my name end up there in the sender email address?
msg2977 (view) Author: montanaro Date: 2015-06-05.12:43:31
FWIW, I have no such problems with bug tracker emails, for this or any other Python-related bug tracker. Gmail seems to always classify them as "ham".
msg2986 (view) Author: Tim.Tisdall Date: 2015-09-03.17:00:03
Here's some headers from a recent email sent to me:

Received-SPF: neutral (google.com: 2a01:4f8:131:2480::3 is neither permitted nor denied by best guess record for domain of roundup-admin@psf.upfronthosting.co.za) client-ip=2a01:4f8:131:2480::3;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: 2a01:4f8:131:2480::3 is neither permitted nor denied by best guess record for domain of roundup-admin@psf.upfronthosting.co.za) smtp.mailfrom=roundup-admin@psf.upfronthosting.co.za;
       dmarc=fail (p=NONE dis=NONE) header.from=python.org

As you can see, "dmarc=fail". (though, I can't seem to see a DMARC policy on any of the pertinent dns records)

Another email from the tracker that was in my spam box said:

"Why is this message in Spam? It's in violation of Google's recommended email sender guidelines."  And linked to https://support.google.com/mail/answer/81126?hl=en-GB#authentication

A DMARC policy should really be added and DKIM signing should also be added.  Lately, major email providers have been clamping down to try to prevent spam by nearly requiring those things.
msg3062 (view) Author: socketpair Date: 2016-07-11.20:37:06
Still no progress? you just required to add SPF-recrod to DNS
msg3066 (view) Author: Tim.Tisdall Date: 2016-07-12.12:54:07
Your comment got emailed to me with no DMARC fail, so that's good.
But adding a policy would be good.

On 11 July 2016 at 16:37, Коренберг Марк
<metatracker@psf.upfronthosting.co.za> wrote:
>
> Коренберг Марк added the comment:
>
> Still no progress? you just required to add SPF-recrod to DNS
>
> ----------
> nosy: +socketpair
>
> _______________________________________________________
> PSF Meta Tracker <metatracker@psf.upfronthosting.co.za>
> <http://psf.upfronthosting.co.za/roundup/meta/issue562>
> _______________________________________________________
History
Date User Action Args
2016-07-12 12:54:07Tim.Tisdallsetmessages: + msg3066
2016-07-11 20:37:06socketpairsetnosy: + socketpair
messages: + msg3062
2015-09-03 17:00:03Tim.Tisdallsetnosy: + Tim.Tisdall
messages: + msg2986
2015-06-05 12:43:32montanarosetnosy: + montanaro
messages: + msg2977
2015-05-09 02:46:48ztanesetmessages: + msg2964
2015-05-09 02:42:10ztanesetnosy: + ztane
messages: + msg2963
2015-02-06 21:03:28fleminrasetmessages: + msg2948
2015-02-06 21:01:47fleminrasetnosy: + fleminra
messages: + msg2947
2015-01-25 17:01:35r.david.murraysetstatus: unread -> chatting
nosy: + r.david.murray
messages: + msg2931
2015-01-25 03:44:16cpburnzcreate