The heart of the matter; the basic essentials; the harsh realities.
Any turn of phrase that is suspected as being racist is subject to close scrutiny, especially in the USA; hence the euphemistic 'N-word'. Reports that a harmless word like picnic originated as the name of a lynching party only have to be voiced to be taken seriously and become part of folk-etymology.
In that context it has been alleged that 'nitty-gritty' is a derogatory reference to the English slave trade of the 18th century. The phrase is usually used with the prefix 'getting down to' and there is a sense that, whatever the nitty-gritty is, it is at the bottom of something. The suggestion is that it originated as a term for the unimportant debris left at the bottom of ships after the slaves had been removed and that the meaning was extended to include the slaves themselves. That report became widely known following newspaper reports of an 'equality and diversity' course for Bristol Council employees in 2005. Had the firm that was conducting the training known that their claims were to reach so wide a public they may have chosen their words more carefully.
The general touchiness over language that might have had a racist origin is enhanced by the ongoing guilt felt by some communities that were formerly involved with the slave trade, for example the English sea-ports of Bristol and Liverpool. In July 2006, Liverpool Council debated the proposal that Penny Lane in Liverpool should be renamed to remove the association with the slave-trader James Penny. This was dismissed as ludicrous by many in the city, but the very fact that the suggestion was made indicates a degree of ongoing unease.
There is no evidence to support the suggestion that 'nitty-gritty' has any connection with slave ships. It may have originated in the USA as an African-American expression, but that's as near as it gets to slavery. It isn't even recorded in print until the 1930s, long after slave ships had disappeared, and none of the early references make any link to slavery.
The first reference that I can find of the phrase in print is from the New York Catalog of Copyright Entries, Part 3 — Musical Compositions, 1937. That lists a song entitled 'That Nitty Gritty Dance' which was copyrighted by Arthur Harrington Gibbs.
The phrase isn't found in print again for some time and reappeared in several newspaper citations in print dating from 1956, for example, this line from Alice Childress' novel Like One of Family:
"You'll find nobody comes down to the nitty-gritty when it calls for namin' things for what they are."
Another is from the Texas newspaper The Daily Journal, in June 1956 and comes from a piece which gave examples of 'the language of 15-year old hepcats':
"She buys, with buffalo heads, ducks to the local flickers, but they prove to be corny and along comes a nitty-grittygator in a cattle train which she hops."
Unfortunately, the Journal didn't include a translation, but I have it on authority of several US contributors of the correct vintage that, in that context, a 'nitty-gritty gator' was a 'lowlife hip dude' and a 'cattle train' was a Cadillac.
It has also been suggested that 'nitty-gritty' refers to head lice, a.k.a. 'nits', or that it refers to ground corn, a.k.a. 'grits', but again, neither of these theories is supported by any hard evidence.
Where it does come from isn't known. It is one of the many phrases that use rhyming reduplication, for example, namby-pamby, willy-nilly etc.