Migrate ProtonMail to GMail
After trying ProtonMail for a year, I decided to migrate my email domain back to Google Workspace from ProtonMail. I like Proton, but in the end my users weren’t happy with the available features, and I was more interested in keeping my users happy.
Anyway, I needed to migrate a corpus of email from ProtonMail to GMail, and all the Internet had to offer was that I should use the Import-Export app to export my mail to mbox, use an extension to load the mbox into Thunderbird, and copy the mail using GMail IMAP.
GMail is great at handling millions of users simultaneously accessing their email, but high throughput from a single user isn’t its strong suit. When I attempted to copy more than 10 messages at a time via IMAP the operation would usually time out.
Here’s a solution for copying email from ProtonMail to GMail that worked better than IMAP for me (for Workspace users this requires imports from webmail hosts to be on):
- Export ProtonMail to a local mbox file using Proton’s Import-Export app.
- Copy the mbox file to a short-lived Cloud VM running a POP3 server.
- Use GMail’s import mail feature to have GMail fetch the messages from your server.
This worked really well. GMail connected every few minutes and pulled 200 messages at a time until all messages were transferred.
Creating a temporary mail server was easier than it sounds. The rough steps I took were:
- Create a GCE Instance running Debian 10. (I used a C3 instance because it was in free public preview, but GMail is conservative in its fetching so I would use an e2-micro next time.)
- Add GCE firewall rules to allow traffic to ports 80 (for Lets Encrypt) and 995 (for POP3 with SSL).
- Assign a DNS record for the host and follow the certbot instructions to obtain a Lets Encrypt certificate.
- Install
dovecot-pop3d, updating the config for the path to the Lets Encrypt SSL certificate and setting the mail_location to~/mbox. - Add a mail user.
Almost surprisingly, that’s all I had to do. The whole process, including setting up the mail import, took around an hour.
Kubernetes e2e tests and feature gates
Today I had to remind myself how the Kubernetes test-infra interacts with features. Unlike with the unit tests, feature gates for the e2e tests are frequently set externally by the CI test definitions rather than the test themselves. Tests that rely on features not set by default are tagged using [Feature:$name] and excluded from the default presubmit tests.
In my case I was adding a test an alpha feature to the e2e node tests. SIG node maintains test configuration that will run tests tagged [NodeAlphaFeature:$name] with --feature-gates=AllAlpha=true, so all I had to do was tag my new tests and remember to set TEST_ARGS="--feature-gates=$name=true" when running locally.
Ephemeral Containers and Kubernetes 1.22
Today we changed the API for Ephemeral Containers in Kubernetes. It’s a setback for those who were hoping for an Ephemeral Containers beta to get the feature enabled in production clusters, but I’m glad we took the time to change it while the feature is still in alpha. The new API use the simpler, well-known pattern that the kubelet uses to update Pod status through a separate subresource. It was quick to implement since it’s actually the same as a prior prototype.
SIG Auth requested the change during the 1.21 release cycle to make it easier for Admission Controllers to gate changes to pods, but my favorite part is that API reference docs will be simpler since we got rid of the EphemeralContainers Kind that was used only for interacting with the ephemeralcontainers subresource.
It’s a large change, though, so the right thing is to hold the revised API in alpha for at least a release to gather feedback. That means the earliest we’d see an Ephemeral Containers beta is 1.23: pretty far from the 1.7 cycle when we started and 1.16 where the feature first landed in alpha. I wonder if that’s a record.
In the mean time, let’s implement all of the feature requests and have nothing left to do in 1.23. Next up: configurable security context.
Ubuntu, systemd-resolver and DVE-2018-0001
I noticed that systemd is spamming syslog with:
Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.
DVE-2018-0001 is a workaround for some captive portals that respond to DNSSEC queries with NXDOMAIN. systemd-resolver in Ubuntu retries every one of these NXDOMAIN responses without EDNS0.
In practice this means one syslog entry every time a domain isn’t resolvable. This is surprising, so I dug further.
Ubuntu pulled in a PR to systemd implementing DVE-2018-0001 in systemd-resolved. It’s not configurable, except that it’s not attempted in DNSSEC strict mode.
As an aside, I feel like Ubuntu integrating unmerged upstream patches isn’t fair to systemd. I incorrectly assumed that it was systemd that was introducing these spammy log messages. Maybe they will eventually, but they haven’t yet.
I’m pretty sure it’s a terrible idea, but I enabled DNSSEC strict mode by setting DNSSEC=yes in /etc/systemd/resolved.conf. I’ll have to try to remember I did this in a few days when I can’t browse the web.
There’s a really good write-up at askubuntu.com of the underlying problem.
Sharing Process Namespace in Kubernetes
Kubernetes pods allow cooperation between containers, which can be powerful, but they have always used isolated process namespaces because that’s all Docker supported at the time Kubernetes was created. This prevented one from doing things like signalling a main process from a logging sidecar, for example.
I’ve been working with SIG Node to change this, though, and Process Namespace Sharing has been released as an Alpha feature in Kubernetes 1.10. Compatibility within an API version (e.g. v1.Pod) is very important to the Kubernetes community, so we didn’t change the default behavior. Instead we introduced a new field in v1.Pod named ShareProcessNamespace. Try it for yourself!
Pods exist to share resources, so it makes sense to share processes as well. I wouldn’t be surprised if process namespace sharing became the default in v2.Pod.
I’d love to hear what you think and whether this feature helps you. Let me know in Kubernetes feature tracking or the comments below.
Debugging regex from the CLI
Just stumbled across this obvious solution from the why didn’t I realize this earlier? department. GNU grep makes an great regex debugger!
Alpine Linux doesn’t work with KubeDNS. Sad.
I was really getting into building docker images from Alpine Linux. I like its philosophy and general 5MB-ness. I discovered tonight, however, that its libc resolver has some significant differences from that of GNU libc. Most notably, the resolver queries all nameservers in parallel and doesn’t support a search path.
I don’t care that much about the search path for these images. Querying the nameservers in parallel sounds great, but unfortunately Kubernetes’ KubeDNS configures a resolv.conf that expects in-order querying. Only the first nameserver will respond with cluster local records.
Oh well, guess I’ll switch everything back over to debian…
Backup to Google Cloud Storage using duplicity 0.6.22
My patch to add support to duplicity for Google Cloud Storage was merged and released with duplicity version 0.6.22. Now backing up to GCS is as easy as backing up to S3. Here are the steps:
- Install duplicity >= 0.6.22.
- Enable Interoperable Access in the Cloud Storage Dashboard.
- Generate Interoperable Storage Access Keys in the Cloud Storage Dashboard.
- Create your bucket:
$ gsutil mb -c DRA gs://BUCKETNAME
The
-c DRAflag enables Durable Reduced Availability for this bucket, which makes sense for backups. - Run
gsutil config -ato generate a~/.botoconfiguration file with your key info. Alternatively (or if you don’t use gsutil) you can set theGS_ACCESS_KEY_IDandGS_SECRET_ACCESS_KEYenvironment variables. - Backup using a
gs://URL. For example:$ duplicity --full-if-older-than 1M --exclude /home/user/.cache \ /home/user gs://BUCKETNAME/backups/user
OSError with duplicity 0.6.19 on OpenBSD and OS X
Sometime around duplicity 0.6.15 (ish) I started running into OSError exceptions that I just didn’t have time to track down. I’ve finally made time, though, and it wasn’t too hard to track down the culprit. I didn’t realize it at the time, but this only affects non-privileged users running duplicity. tl;dr choose a different TMPDIR.
tech stuff. 
You must be logged in to post a comment.