Do not use windows line endings

Recently, our team at work stumbled upon a strange behavior in their Git projects: there were files with Windows CRLF line endings checked into the git repository. If you know git, you’re aware of the line ending settings and that line endings basically should never be an issue. Well, turns out: it is, and it is quite often.

How it should work

While investigating the issue, I found the excellent Mind the End of Your Line article over at adaptivepatchwork.com, which explains how line endings should be handled in the first place. It also helped me find the cause of the issue, so props to them!

The most important bit in this case is the core.autocrlf setting. As every git setting, it can be specified per-repository or globally (per OS user). On Windows, it seems to be recommended setting it to true, causing git to use Windows CRLF in the workspace and auto-converting line endings to LF when committing to git’s object database. Going with the input setting also shouldn’t cause too much trouble: in this case, git checks line endings out into the workspace as they are in the object database without changing them, but still makes sure to replace any Windows CRLF line endings with linux LF when committing to git’s object database.

The only core.autocrlf setting to avoid at all cost, at least on windows, is false. This would cause git to commit line endings into its object database exactly as they appear in the workspace (the only setting letting Windows’ CRLF into git’s object database).

Our setup

We are mostly dealing with java projects, but our repositories contain a bunch of shell scripts for automation, testing and stuff like that. Our development machines are Windows workstations while test and production servers are linux/unix. We use Eclipse along with its egit plugin for development.

The symptom

I once had to merge a huge feature branch. After resolving merge conflicts and committing my changes, I noticed that pushing the merge took longer than usual. Also, the git repository had nearly doubled in size. It turned out that my merge just changed every single text file’s line endings to Windows CRLF in the repository.

I fixed it by resetting everything to linux LF line endings. However, after a couple of months, we noticed that Windows line endings were slowly crawling back into the repository – apparently cumulating through multiple developers’ commits and merges over time.

Pinpointing the problem

This turned out to be a nightmare. It took an awful lot of time to reproduce the problem, let alone find a logical explanation to why this kept happening.

Whenever I ran git config core.autocrlf in a repository, it always yielded the expected input, so I didn’t suspect the autocrlf option and didn’t care that git config --global core.autocrlf wasn’t set. I ended up creating a test git repo and running a bunch of merges with intentional conflicts before I was able to pinpoint the exact constellation causing the problem:

As it turns out, it is a combination of the git spec, the windows git command-line and Eclipse’s egit plugin causing the issue.

• The git documentation states that settings are either found in the current repository’s ./.git/config file or the global %HOME%.gitconfig, the latter being the fallback if they’re not found in the former. It also specifies that the default value for the core.autocrlf setting is false – remember? the worst setting there could be on Windows if line endings matter…
• The Windows git client (from git-scm.com) does ask for the core.autocrlf setting during setup to make sure you have it set, but it doesn’t seem to store it in the documented global location (%HOME%.gitconfig). Instead, it seems to store it somewhere else (I haven’t found where though) and uses this location as ultimate fallback during command-line git invocations when the setting isn’t found in the usual locations. As long as you only ever use the command-line git, you’re fine, however…
• Eclipse’s egit plugin doesn’t use the command-line git client directly but comes with its own implementation, which does honor the documented locations and settings. However, as the “magical ultimate fallback” used by the Windows git command-line client seems to be undocumented, Eclipse couldn’t possibly know about it. So from Eclipse’s point of view: No core.autcrlf in the repository, no core.autocrlf in %HOME%.gitconfig: use the default value: false. If you then happen to commit merges, or even worse, if you previously cloned a repository with core.autocrlf set to true, your Eclipse will end up committing everything with Windows line endings while your command-line git behaves perfectly fine.

I checked with some colleagues from other teams and they also had the global core.autocrlf setting unset – and incidentally, their Eclipse was behaving strangely in the git synchronize view. After they set the global setting, everything was fine. So it looks like manual action is needed to make it work correctly.

Oh, and to make things even worse: I currently suspect something to overwrite my %HOME%.gitconfig, removing the core.autocrlf setting as it magically disappeared somewhere in the last weeks. I’m not quite sure what it is tough…

In conclusion, TL;DR

If you use Eclipse and Git on Windows, make sure you have the core.autocrlf option globally set to true AT ALL TIMES!

To check for the option:

    git config --global core.autocrlf

(this should yield true).

To set it:

    git config --global core.autocrlf true

May this post help other poor souls like me! 😉

– Cheers!

Git tries to help translate line endings between operating systems with different standards. This gets sooo frustrating. Here’s what I always want:

On Windows:

git config --global core.autocrlf input
This says, “If I commit a file with the wrong line endings, fix it before other people notice.” Otherwise, leave it alone.

On Linux, Mac, etc:

git config --global core.autocrlf false
This says, “Don’t screw with the line endings.”

Nowhere:

git config --global core.autocrlf true
This says, “Screw with the line endings. Make them all include carriage return on my filesystem, but not have carriage return when I push to the shared repository.” This is not necessary.

Windows and Linux on the same files:

This happens when you’re running Linux in a docker container and mounting files that are stored on Windows. Generally, stick with the Windows strategy of core.autocrlf=input, unless you have .bat or .cmd (Windows executables) in your repository.

The VS Code docs have tips for this case. They suggest setting up the repository with a .gitattributes file that says “mostly use LF as line endings, but .bat and .cmd files need CR+LF”:

* text=auto eol=lf
*.{cmd,[cC][mM][dD]} text eol=crlf
*.{bat,[bB][aA][tT]} text eol=crlf

If VSCode insists on putting CR (pictured as ^M in the git diff) in a file, then open the file and check the lower right-hand corner, in the status bar. Does it say “CRLF”? Click that and choose “LF” instead. Do things right, VSCode 😠

Troubleshooting

When git is surprising you:

Check for overrides

Within a repository, the .gitattributes file can override the autocrlf behavior for all files or sets of files. Watch out for the text and eol attributes. It is incredibly complicated.

Check your settings

To find out which one is in effect for new clones:
git config --global --get core.autocrlf

Or in one repository of interest:
git config --local --get core.autocrlf

Why is it set that way? Find out:
git config --list --show-origin
This shows all the places the settings are set. Including duplicates — it’s OK for there to be multiple entries for one setting.

Why does this even exist?

Historical reasons, of course! (If you have a Ruby Tapas subscription, there’s a great little history lesson on this.)

Back in the day, many Windows programs expected files to have line endings marked with CR+LF characters (carriage return + line feed, or \r\n). These days, these programs work fine with either CR+LF or with LF alone. Meanwhile, Linux/Mac programs expect LF alone.

Use LF alone! There’s no reason to include the CR characters, even if you’re working on Windows.

One danger: new files created in programs like Notepad get CR+LF. Those files look like they have \r on every line when viewed in Linux/Mac programs or (in code) read into strings and split on \n.

That’s why, on Windows, it makes sense to ask git to change line endings from CR+LF to LF on files that it saves. core.autocrlf=input says, screw with the line endings only in one direction. Don’t add CR, but do take it away before other people see it.

Postscript

I love ternary booleans like this: true, false, input. Hilarious! This illustrates: don’t use booleans in your interfaces. Use enums instead. Names are useful. autocrlf=ScrewWithLineEndings|GoAway|HideMyCRs