Mfcttrf

When did Unix move away from storing clear text passwords in passwd? Also, when was the shadow file introduced?

For the early history of Unix password storage, read Robert Morris and Ken Thompson's Password Security: A Case History. They explain why and how early Unix systems acquired most the features that are still seen today as the important features of password storage (but done better).

• The first Unix systems stored passwords in plaintext. Unix Third Edition introduced the crypt function which hashes the password. It's described as “encryption” rather than “hashing” because modern cryptographic terminology wasn't established yet and it used an encryption algorithm, albeit in an unconventional way. Rather than encrypt the password with a key, which would be trivial to undo when you have the key (which would have to be stored on the system), they use the password as the key.


• When Unix switched from an earlier cipher to the then-modern DES, it was also made slower by iterating DES multiple times. I don't know exactly when that happened: V6? V7?


• Merely hashing the password is vulnerable to multi-target attacks: hash all the most common passwords once and for all, and look in the password table for a match. Including a salt in the hashing mechanism, where each account has a unique salt, defeats this precomputation. Unix acquired a salt in Seventh Edition in 1979.


• Unix also acquired password complexity rules such as a minimum length in the 1970s.

Originally the password hash was in the publicly-readable file /etc/passwd. Putting the hash in a separate file /etc/shadow that only the system (and the system administrator) was one of the many innovations to come from Sun, dating from around SunOS 4 in the mid-1980s. It spread out gradually to other Unix variants (partly via the third party shadow suite whose descendent is still used on Linux today) and wasn't available everywhere until the mid-1990s or so.

Over the years, there have been improvements to the hashing algorithm. The biggest jump was Poul-Henning Kamp's MD5-based algorithm in 1994, which replaced the DES-based algorithm by one with a better design. It removed the limitation to 8 password characters and 2 salt characters and had increased slowness. See IEEE's Developing with open source software, Jan–Feb. 2004, p. 7–8. The SHA-2-based algorithms that are the de facto standard today are based on the same principle, but with slightly better internal design and, most importantly, a configurable slowness factor.

I am not a big fan of copy-paste so have a look at the history section in the wikipedia's passwd page ;)

EDIT: As it seems it's not a good practice to just share a link with the exact information that you want. So apart from the link let me also quote the wikipedia's relevant section here:

Password shadowing first appeared in Unix systems with the development
of SunOS in the mid-1980s,[10] System V Release 3.2 in 1988 and BSD4.3
Reno in 1990. But, vendors who had performed ports from earlier UNIX
releases did not always include the new password shadowing features in
their releases, leaving users of those systems exposed to password
file attacks.

System administrators may also arrange for the storage of passwords in
distributed databases such as NIS and LDAP, rather than in files on
each connected system. In the case of NIS, the shadow password
mechanism is often still used on the NIS servers; in other distributed
mechanisms the problem of access to the various user authentication
components is handled by the security mechanisms of the underlying
data repository.

In 1987 the author of the original Shadow Password Suite, Julie Haugh,
experienced a computer break-in and wrote the initial release of the
Shadow Suite containing the login, passwd and su commands. The
original release, written for the SCO Xenix operating system, quickly
got ported to other platforms. The Shadow Suite was ported to Linux in
1992 one year after the original announcement of the Linux project,
and was included in many early distributions, and continues to be
included in many current Linux distributions.

I don't yet have a primary source, but according to this TrustedSec post (emphasis mine):

Early systems stored passwords in plaintext but eventually this was
replaced by more secure forms of password storage. Robert Morris
developed crypt based on the m-209 cipher machine and it appeared in
Version 3 Unix, though Crypt was not used to store passwords until 6th
Edition Unix (1974).

According to multiple sources, Version 3 UNIX was released in February, 1973.

From the original paper by Thompson and Morris, we can confirm that plaintext storage was originally used:

The UNIX system was first implemented with a password file that
contained the actual passwords of all the users, and for that reason
the password file had to be heavily protected against being either
read or written.

/etc/shadow appeared in multiple branches of UNIX as noted in other answers.