[Solved] dovecot login problem after 2.4 update

[pogwizd]

Hi,

I run on a PC a mailserver with postfix and dovecot, with no problems.
When dovecot 2.4 arrived, I postponed the update, due to the configuration breaking, but now, I want to migrate my server to a new machine, so of course, I will step straight forward with the new version.

Doing so, i am following the following how to: Complete Virtual Mail Server

I understand that this wiki page is to be updated Complete Virtual Mail Server/Dovecot to Database (I'll try to contribute to it when I manage to make the whole thing work), but tried to adapt the configuration file, and as you can imagine. it fails.

Here is my dovecot.conf file for the moment.
I know it would look nicer to have everything in seperate files, but I will do it once the whole thing works.

When I try to login, I have the following result:

Code: Select all

root@MyPC ~ # dovecot auth login MyUser@MyDomain.com
Password:
passdb: MyUser@MyDomain.com auth failed
extra fields:
user=MyUser@MyDomain.com
original_user=MyUser@MyDomain.com
root@MyPC ~ #

Now, as for the logs, it does not give much either as:

• The dovecot-debug.log is empty
• The dovecot-error.log is empty
• The dovecot-info.log has just one message when attempting to log in:

  Nov 21 10:13:36 auth(?,sasl:plain): Info: Empty username

  I was wondering if someone may have an idea on what it is that I missed in my configuration file?
  I would guess something with the pgsql queries, but what?
  
  Thanks in advance.

[alamahant]

Is dovecot built with USE="argon2"?

[pogwizd]

Yes it is.

Code: Select all

~ $ emerge dovecot -p

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 2.46 s (backtrack: 0/20).

[ebuild   R    ] net-mail/dovecot-2.4.2:0/2.4.2::gentoo  USE="argon2 kerberos ldap lua lz4 pam postgres sqlite systemd zstd -cdb -managesieve -mysql (-selinux) -sieve -solr -static-libs -stemmer -suid -system-icu -test -textcat -unwind -xapian" LUA_SINGLE_TARGET="lua5-4 -lua5-3" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

* IMPORTANT: 33 news items need reading for repository 'gentoo'.
* Use eselect news read to view new items.
~ $

[alamahant]

I was dreading also the transition to 2.4 until I found this :
https://www.server-world.info/en/note?o ... p=mail&f=1
But in your case I guess its the postgresql backend and/or the argon scheme.
Maybe for testing purposes try one user with cram md5?

[pogwizd]

I tried it, but I still get the same.

But in your case I guess its the postgresql backend

Yes, that is my guess also that something is wrong now with the queries, especially when reading the error:

Info: Empty username

[grknight]

Code: Select all

  query = SELECT local_part AS username, domain, password FROM mailbox WHERE local_part = '%{user}' AND domain = '%{domain}'

According to the documentation, the user variable is in the form user@domain.

Correct me if I am wrong, but what I think you really want is:

Code: Select all

  query = SELECT local_part AS username, domain, password FROM mailbox WHERE local_part = '%{user|username}' AND domain = '%{user|domain}'

The filters split the user into pieces and may be what you are looking for.

Edit: this is reinforced with examples in another section of the documentation.

[pogwizd]

Thanks @grknight for the advice.

I took it into consideration and changed the query within my dovecot.conf file.
Nevertheless, it unfortunetelly still does not work.

I tried to play with those query, but with no result.
What is strange, is that when I manually enter those query within pgsql, I have the answers.

Maybe I understood something wrong with those queries. When I do it, I change of course the variables with meaningfull values like the following:

• %{user|username} with MyUser@MyDomain.com
• %{user|domain} with MyDomain.com

Just to be clear, the results look like this:

Code: Select all

SELECT username AS username, domain,  password AS password FROM mailbox WHERE username = 'MyUser@MyDomain.com' AND domain = 'MyDomain.com';
username                       |          domain           |                                                  password
----------------------------------+---------------------------+---------------------------------------------------------------------------------------------
MyUser@MyDomain.com | MyDomain.com | {ARGON2ID}$argon2id$v=19$m=65536,t=3,p=1$PeFEoYix5lBO5eFI8DJwng$PWPfTLAnM4cJbNLXNzsKygDE0c/9LYUEgaDGY+6pF6U
(1 row)

What I noticed is that the local_part contains only the MyUser information.

But whatever I tested, I still had the following error:

Code: Select all

Nov 21 23:50:59 auth(?,sasl:plain): Info: Empty username

And this I cannot explain, as I am quite sure the queries should return something.

[grknight]

Code: Select all

SELECT username AS username, domain,  password AS password FROM mailbox WHERE username = 'MyUser@MyDomain.com' AND domain = 'MyDomain.com'

And this I cannot explain, as I am quite sure the queries should return something.

If you require the values like this, then this is the correct query:

Code: Select all

query = SELECT username, domain, password FROM mailbox WHERE username = '%{user}' AND domain = '%{user|domain}'

[pogwizd]

Thanks a lot @grknight,
Your help was very helpfull, and I finally get it working.

So indeed, the query was one of the problems, and three of them are working properly in fact. Those are the three in question:

Code: Select all

#   query = SELECT local_part AS username, domain, password FROM mailbox WHERE local_part = '%{user | username}' AND domain = '%{user | domain}'
  query = SELECT local_part AS user, domain, password FROM mailbox WHERE local_part = '%{user | username}' AND domain = '%{user | domain}'
#   query = SELECT username, domain, password FROM mailbox WHERE username = '%{user}' AND domain = '%{user|domain}'

Obviously, I kept only the uncommented one.

In order to make it work, I enable the maximum debugging option I could find with the following:

Code: Select all

log_debug = category=ssl
log_debug = category=auth
auth_debug = yes
mail_debug = yes

And as you can imagine, there was of course something else to change.

The last piece of the puzzle was the auth_username_format.
I changed it like so:

Code: Select all

auth_username_format = %{user | username}@%{user | domain}

And voila.

Again, thanks to everyone.

[Frautoincnam]

@pogwizd
Could you please provide your final configuration again? The link you gave is broken.

[pogwizd]

@Frautoincnam

Here you go: my dovecot config
Please note that here, everything is in one file, but technically, I split it into several files within the conf.d folder...

[Frautoincnam]

@Frautoincnam

Here you go: my dovecot config
Please note that here, everything is in one file, but technically, I split it into several files within the conf.d folder...

Thank you. This will be a good starting point for my migration.

[Akaihiryuu]

What's a good alternative to Dovecot? I'm going to drop it, I'm not going to jump through all of these hoops just because they wanted to change how everything works for no good reason.

[nicop]

The update is not that difficult and options are somewhat the same. But it’s about mail, so nobody likes failure on restart.

What's a good alternative to Dovecot?

courier / cyrus (not uptodate)
But starting again with a new one will take much more time

[Akaihiryuu]

The update is not that difficult and options are somewhat the same. But it’s about mail, so nobody likes failure on restart.

What's a good alternative to Dovecot?

courier / cyrus (not uptodate)
But starting again with a new one will take much more time

Dovecot seems to be forcing vmail with virtual users. I DO NOT want this. I want the same simple maildir with the mail stored in user's home directories. This is just a simple local email server, and I do not want anything more than that. I don't want all this complicated junk, I just want it to work like it did before. Is it still possible to do this? If not I have to drop Dovecot and use something else. I am NOT setting up vmail or virtual users.

[nicop]

Dovecot doesn't force anyone, system users is still available: https://doc.dovecot.org/2.4.2/core/conf ... ystem.html
But please post your conf from 2.3

[Ralphred]

I want the same simple maildir with the mail stored in user's home directories. This is just a simple local email server, and I do not want anything more than that.

I literally finished my upgrade with the same use case a couple days ago.

• There were only a few sticking points really:
• The 2.4.2 documentation makes use of the ~/maildir in it's actual examples, however dovecot will start complaining about relative paths; switching to /home/%{user}/.maildir (.maildir is what I use) is accepted and works as expected
• Kind of related to above, the the documentation implies that

  Code: Select all

  mail_home = /home/${user}/
  mail_path = .maildir

  Will work, despite trying to create a .maildir in / with this config, just use

  Code: Select all

  mail_home = /home/${user}/
  mail_path = /home/${user}/.maildir

• mail_uid= and mail_gid= : I garnered many opinions on about how to make 2.4 function similarly to 2.3 in this use case, none of which worked except my solution of

  Code: Select all

  mail_gid = %{user}
  mail_uid = %{user}

  also, all other solutions left users mail exposed in one way or another.

I had to use

Code: Select all

  inet_listener imap {
    port = 0
  }

in the appropriate place to prevent it listening on 143, but that's the same as 2.3 so...

[Akaihiryuu]

I want the same simple maildir with the mail stored in user's home directories. This is just a simple local email server, and I do not want anything more than that.

I literally finished my upgrade with the same use case a couple days ago.

• There were only a few sticking points really:
• The 2.4.2 documentation makes use of the ~/maildir in it's actual examples, however dovecot will start complaining about relative paths; switching to /home/%{user}/.maildir (.maildir is what I use) is accepted and works as expected
• Kind of related to above, the the documentation implies that

  Code: Select all

  mail_home = /home/${user}/
  mail_path = .maildir

  Will work, despite trying to create a .maildir in / with this config, just use

  Code: Select all

  mail_home = /home/${user}/
  mail_path = /home/${user}/.maildir

• mail_uid= and mail_gid= : I garnered many opinions on about how to make 2.4 function similarly to 2.3 in this use case, none of which worked except my solution of

  Code: Select all

  mail_gid = %{user}
  mail_uid = %{user}

  also, all other solutions left users mail exposed in one way or another.

I had to use

Code: Select all

  inet_listener imap {
    port = 0
  }

in the appropriate place to prevent it listening on 143, but that's the same as 2.3 so...

This doesn't work. Their documentation is so laughably incomplete it may as well not even be there.

Code: Select all

Jan 17 11:24:32 triforce dovecot[1148]: imap-login: Logged in: user=<stormspark>, method=PLAIN, rip=192.168.0.2, lip=192.168.0.1, mpid=1158, TLS, session=<AOiR5JdI5dLAqAAC>
Jan 17 11:24:32 triforce dovecot[1148]: imap(stormspark): Error: Unknown UNIX GID group: stormspark (from mail_gid setting)

Code: Select all

## Dovecot configuration file

# Dovecot configuration version. This must be the first setting in the
# configuration file. It specifies the configuration syntax, the used setting
# names and the expected default values.
dovecot_config_version = 2.4.2

# Dovecot storage file format version. It specifies the oldest Dovecot version
# that must be able to read files written by this Dovecot instance. The
# intention is that when upgrading Dovecot cluster, this setting is first kept
# as the old Dovecot version. Once the cluster is fully upgraded to a new
# version and there is no intention to rollback to the old version anymore,
# this version number can be increased.
dovecot_storage_version = 2.4.2

# The configuration below is a minimal configuration file using system user authentication.
# See https://doc.dovecot.org/latest/core/config/quick.html

!include_try conf.d/*.conf

# Enable wanted protocols:
protocols {
  imap = yes
  lmtp = yes
}

mail_home = /home/${user}/
mail_driver = maildir
mail_path = ~/.maildir

mail_uid = %{user}
mail_gid = %{user}

# By default first_valid_uid is 500. If your vmail user's UID is smaller,
# you need to modify this:
first_valid_uid = 8

namespace inbox {
  inbox = yes
  separator = /
}

# Authenticate as system users:
passdb pam {
}

ssl_server {
  cert_file = /etc/dovecot/server.pem
  key_file = /etc/dovecot/server.key

I haven't yet found anything else that will work for me...the moment I do, Dovecot is being nuked off of my system permanently. It's absolutely terrible software.

[Ralphred]

Code: Select all

mail_path = ~/.maildir

Mate, I literally covered this.
Would you like the whole config, I run just imaps though, nothing else?

[Akaihiryuu]

Code: Select all

mail_path = ~/.maildir

Mate, I literally covered this.
Would you like the whole config, I run just imaps though, nothing else?

I just figured it out. I had tried yours and then ran through a bunch of other stuff before I posted the config. mail_path and mail_home need to be %{user} and not ${user}

Here's what finally worked for me (I had to change a couple other things):

Code: Select all

## Dovecot configuration file

# Dovecot configuration version. This must be the first setting in the
# configuration file. It specifies the configuration syntax, the used setting
# names and the expected default values.
dovecot_config_version = 2.4.2

# Dovecot storage file format version. It specifies the oldest Dovecot version
# that must be able to read files written by this Dovecot instance. The
# intention is that when upgrading Dovecot cluster, this setting is first kept
# as the old Dovecot version. Once the cluster is fully upgraded to a new
# version and there is no intention to rollback to the old version anymore,
# this version number can be increased.
dovecot_storage_version = 2.4.2

# The configuration below is a minimal configuration file using system user authentication.
# See https://doc.dovecot.org/latest/core/config/quick.html

!include_try conf.d/*.conf

# Enable wanted protocols:
protocols {
  imap = yes
  lmtp = yes
}

mail_driver=maildir
mail_home = /home/%{user}/
mail_path = /home/%{user}/.maildir

mail_uid = %{user}
mail_gid = users

# By default first_valid_uid is 500. If your vmail user's UID is smaller,
# you need to modify this:
first_valid_uid = 8

namespace inbox {
  inbox = yes
  separator = /
}

# Authenticate as system users:
passdb pam {
}

ssl_server {
  cert_file = /etc/dovecot/server.pem
  key_file = /etc/dovecot/server.key

[Ralphred]

need to be %{user} and not ${user}

My apologies, too much bash, yes it's % not $.

[Akaihiryuu]

need to be %{user} and not ${user}

My apologies, too much bash, yes it's % not $.

Oh no problem not your fault...they don't have decent documentation and their example.conf is horibly lacking.