In ancient Rome, the related punishment of infamia could be imposed on criminal offenders. In this case, the principle penalties were loss of suffrage and the right to serve in the Roman legions (a desired opportunity). [...]

In medieval Europe, the legal doctrines of 'civil death' and 'outlawry' carried forward similar notions. As with atimia, those punished with civil death generally suffered a complete loss of citizenship rights (in some early Germanic texts, outlaw status meant a 'loss of peace' that was comparable to becoming a wolf, since the outlaw had to 'live in the forest'). In extreme cases, civil death could be injurious or fatal, since outlaws could be killed by anyone with impunity, or have their property seized. In most medieval contexts, political rights held little substantive meaning. But the civil death model carried over into parts of modern criminal law."
2006 Jeff Manza and Christopher Uggen   Locked Out: Felon Disenfranchisement and American Democracy

[T]he revolutionary period [...] witnessed heated public exchanges and sharp political conflict over the [voting] franchise. [...] Implicit in these arguments was the claim that voting was not a right but a privilege, one that the state could legitimately grant or curtail in its own interest. [...]

Yet there was a problem with this vision of suffrage as a right [...] there was no way to argue that voting was a right or a natural right without opening a Pandora's box. If voting was a natural right, then everyone should possess it. [...]

[S]everal important legal and jurisdictional issues also were shaped, or structured, during the revolutionary period. The first was that suffrage was defined as a constitutional issue [...]  Implicit in this treatment was the notion that suffrage requirements ought to be durable and difficult to change."
2000 Alexander Keyssar  The Right to Vote

By making the franchise in national elections dependent on state suffrage laws, the authors of the Constitution compromised their substantive disagreements to solve a potentially explosive political problem. [...] citizenship in the new nation -- controlled by the federal government -- was divorced from the right to vote."
2000 Alexander Keyssar   The Right to Vote

"Section 1. The right of citizens of the United States to vote shall not be denied or abridged by the United States or by any State on account of race, color, or previous condition of servitude.

Rhetorically and politically [...] the Civil Rights Act of 1957 did push the voting rights agenda forward, largely through the creation of the Commission on Civil Rights (CCR) [...] The three Northerners on the commission, including its chairman, went further, calling for a new constitutional amendment that 'would give the right to vote to every citizen who meets his State's age and residence requirement and who is not legally confined at the time of registration or elections' [...]

The Civil Rights Commission thus lent its prestige and authority to calls for further federal action and the de facto, if not de jure, nationalization of the right to vote."
2000 Alexander Keyssar The Right to Vote: The Contested History of Democracy in the United States

Among its other provisions, the Act contained special enforcement provisions targeted at those areas of the country where Congress believed the potential for discrimination to be the greatest. Under Section 5, jurisdictions covered by these special provisions could not implement any change affecting voting until the Attorney General or the United States District Court for the District of Columbia determined that the change did not have a discriminatory purpose and would not have a discriminatory effect. In addition, the Attorney General could designate a county covered by these special provisions for the appointment of a federal examiner to review the qualifications of persons who wanted to register to vote. Further, in those counties where a federal examiner was serving, the Attorney General could request that federal observers monitor activities within the county's polling place."

Although the Court has never given plenary consideration to the precise question of whether a State may constitutionally exclude some or all convicted felons from the franchise, we have indicated approval of such exclusions on a number of occasions.... recently we have strongly suggested in dicta that exclusion of convicted felons from the franchise violates no constitutional provision....

Especially striking for our purposes is the February 2002 U.S. Senate vote on an amendment to the federal voting reform legislation that proposed to restore voting rights to ex-felons in federal elections. Senators from the 11 former confederate states voted 18 to 4 against enfranchisement (the measure went down by a 63-31 floor vote), and the most passionate speeches against it were made by southerners..."
2006 Jeff Manza and Christopher Uggen   Locked Out: Felon Disenfranchisement and American Democracy

In its Decision, the Court concluded that it is 'compelled to find that there is discrimination in Washington’s criminal justice system on account of race' and that this discrimination 'clearly hinders the ability of racial minorities to participate effectively in the political process.' Despite these conclusions, however, the Court dismissed the case citing a 'remarkable absence of any history of official discrimination' in Washington’s electoral process and felon disenfranchisement provisions."
6/18/05 The Sentencing Project