Debugging --enable_sendmail on AppEngine SDK for Mac OSX
Google AppEngine's SDK comes with an --enable_sendmail flag. However, if messages do not actually send, the GAE docs do not really help you figure out why. This post should give you a few pointers on how to diagnose and a solution for the specific case of a blocked port 25.
Quick Solution for Blocked Port 25 Issue
If you think/know that your ISP is blocking port 25, then you have two options:
- Stop using --enable_sendmail and use the --smtp_* params to supply your smtp username, password etc for every time you fire up the dev_appserver
- Set your machine's email system config to use smtp by default on secure port 587 (How to Do This for OSX)
I did the later option. It is more fiddling with low leve stuff that I'd like, but I also don't need to have my email password being passed to GAE.
How To Tell if your Port 25 is Blocked
Simply put, if sending an email times out, then the port is likely blocked. You can can see this via the mail system error logs or a telnet command.
I'm assuming this is also an issue for sending email with local email clients (Thunderbird, etc). If you have one of these installed, and it works to send mail, check the preferences for which port it is sending messages through.
Here's an overview of the Situation after debugging
- AppEngine's mail stub was working just fine. Messages were stuck in the mailqueue w/o any GAE level warnings. As far as GAE knew, they were sent.
- The error logs in /var/log/mail.log contained lots of timeout errors
- My ISP, XFinity/Comcast, recently started blocking port 25 (outgoing email port) - Read More Why. Note: The extensive list of ISPs also blocking this port.
- I had to set up my system email to use port 587 which is secure. Read How To Do This for OSX. In the comments there are a few tweaks for OSX Yosemite specifically. Once I did these I was all set. Note: That I had to use my credentials for my personal email as my work email uses SAML (I'm working on figuring this out).
The Background
I recently received a new Macbook Pro from work and had to rebuild my development environment. I thought I had everything set up perfectly, until I attempted to debug an issue that required setting up a new account in the product. Our registration flow is fairly common in that it sends out a welcome email containing a temp password and activation link. If you do not receive the email, you cannot log in to the product. This is mostly a non issue when deployed to the appspot, but when trying to debug something locally, it can be a pain in the butt... especially when AppEngine doesn't send the email. I could have easily just logged out the contents of the email sent off to GAE's internal methods, but I wanted to actually solve this issue.
How I Arrived at This Solution
Initially, I assumed something was wonky with the AppEngine setup. Most of my searching revealed that the --enable_sendmail flag exists and also bits about AppEngine's FROM: address criteria when sending email on an appspot. It wasn't until I ran into a post unrelated to GAE about checking the system mail logs for fail notices. When I opened up /var/log/mail.log, I saw lots of timeouts attempting to connect to ASPMX.L.GOOGLE.com. I quickly realized, the google.com had nothing to do with AppEngine and everything to do with the email address I was attempting to send to was on an domain hosted by GoogleApps (i.e. my work email address). That said, I found this article that suggested trying to telnet into ASPMX.L.GOOGLE.com on port 25. For me it timed out. I had a coworker attempt it, and it worked and was instant for them.
Now that I started to think, it was something with the port and/or my ISP (XFinity/Comcast), I found this article saying that, yes indeed, XFinifty and a number of other ISPs block port 25 but allow for the secure port 587. From there, it was just a matter of setting up my mail system to use that by default. It isn't pretty or easy per se, but it is outlined heavily in this post.
In the End
Sending email is working again for me. Since doing all this, I did notice that the dev_appserver does support smtp host/user params, which is essentially what we set up before. That said, I like the idea of email being to work by default again w/o having to pass the auth params in each time.
As per usual, I hope this helps someone save a few hours of head scratching and searching dead ends. Enjoy.
Helpful Commands
mailq - Shows the contents of the mail queue
date | mail -s "Testing 1 2 3" user@example.com - Sends an email message with subject "Testing 1 2 3" to user@example.com. Body of the email is the current date.
sudo postfix reload - Reload the postfix daemon when doing any settings changes
sudo postsuper -d ALL - Remove all messages from the mail queue - check it by running mailq
sudo pico /etc/postfix/main.cf
open -f /var/log/mail.log - Opens the mail log. For me, this opens in OSX's nice Log viewer program. You can also use -tail