Despair, Teshuvah, and Hope.

Sometimes, a person is convinced that he is so sinful that teshuva won’t help. This state of despair can mean one of two things. One: that he is, indeed, worthless, or Two: that he is not.
Let's say he is indeed an irredeemable rasha. Well, that’s not a exemption from doing avodas Hashem. It can even be liberating– you don’t have to bother trying to repair what you’ve done wrong, but there’s no reason not to do whatever good you can, bein odom lechaveiro or lamokom, in the time you have. But most likely the possibility of proper t’shuva still exists, but he is denying it– it’s an avoidance mechanism–because he doesn’t want to give up his irresponsible way of life.

(We lie to ourselves all the time. I am reminded of this whenever someone dies, and a relative says "If only I had taken the opportunity to call them more often, take care of them better, kiss them, tell them I love them, sit and learn from them......" on and on. What this really means is that the person knew, deep in his heart. that he could have been doing this all along, but decided he didn't want to. Only when the relative is safely dead can he say "If I had been more thoughtful I would have done a, b, c...." No. You're lying to yourself. You didn't get wiser because the other person died. You never wanted to do these things and you never would have done these things.)

Someone pointed out to me that Reuven’s teshuva did not regain for him what could have been his. His malchus and kehuna were irretrievably gone– pachaz kamayim, ahl tosar. Period. Similarly, Adam’s delayed teshuva did not get him back into gan eden, the dohr hamidbar did teshuva for the chet hameraglim and the chet ha’eigel, but the gzeiras hamidbar stood; and Shaul did teshuva for his chet with Amaleik but lost his malchus anyway, and so on. So Acher might have been right about the futility of his attempting teshuva.

On the other hand, we find that Menashe’s teshuva did enable him to retain his malchus-- and his aveiros were much more serious than most of the aforementioned.

As a consolation, the Chasam Sofer here does say that "yeser se'eis ve'yeser oz" means that Reuven's position as King of Baalei Teshuva is greater than his previous status as progenitor of Malchus and Kehuna, because bemakom....

Things to consider on this topic:

I think that one of the factors here is that if before teshuva the person’s rights/position were given to someone else, or a set of circumstances was set into motion, it is not revoked. You can’t be sho’eil on hefker once someone else has taken the object, and you can't be shoeil on hekdesh once the Kohen has the korban/trumah.

Not regaining previous entitlements is not the same as not being forgiven.

When you look at Reuven’s case, you also have to see the obvious contrast with Yehuda, whose teshuva seems to have regained him whatever he had previously lost. Adam's teshuva did not help, but Kayin's did. We know that it is possible to completely abandon a particular chet without changing the middah that caused it (Prishus/Nekius). Maybe the difference is whether there was just charatah/azivas hachet or also the actual changing of the middah that caused the chet.

In parshas Vayechi, in Breishis 49:4, on Pachaz kamayim, Reb Elya Lopian says that the loss of malchus and kehuna was not an onesh for the chillul yetzu’ei aviv; as Rashi points out, pachaz here is a noun, not a verb, so the accent is at the beginning. The chilul showed how he allowed his middah of 'pachaz' to come out, and the middah is the reason he could not have malchus or kehunah. He adds, you can’t say it was a real punishment, because the Torah and Chazal stress how Reuven did teshuva. It must be that it was not an onesh, but rather a necessary result of the fact of the middah ra’ah that was inconsistent with kehunah and malchus.

He brings a Chofetz Chayim that says that laundering clothes can take out stains, but it doesn’t sew on new buttons. The idea is that teshuva is a good thing, and it does remove the p’gam of the aveiroh, but there might be a middoh ra’ah that has to be removed, and teshuva is not going to remove a middah ra’ah. That takes a lot of separate work.

I saw the magi’ah in the Shelah in the later section on Teshuva, toward the end of the sefer, that says (with my additions) that a person should be careful not to treat symptoms, but to find and cure the underlying cause, both in the case of health problems and spiritual problems. A person goes to the doctor with a rash, or a cough. The doctor might look at the rash, and say, yes, you have a rash, and prescribe a lotion, or prescribe a cough suppressant for the cough. This is foolish, because he is only treating the symptom. A better doctor will look further, and determine what is causing the rash or the cough, what the underlying problem is. A wiser doctor would not rest at that point. He would try to investigate and to find out why this person caught this illness. Maybe he has mold in the walls of his house. Maybe he has a friend who is a carrier, from whom he caught the disease. He might find the source of the fungus, or a reservoir of bacteria. And even worse— the patient might have systemic condition of depressed immunity— that he catches what others would shrug off easily. In the context of teshuva, this would be called a lack of character. The lesson is that the same way a good doctor will treat the symptom, the underlying cause, and the weakened immunity, a baal teshuvah must treat the act of sin, the underlying taivah, and the failure of yiras shamayim and bitachon.

See the Tanchuma on the passuk by Reuven that says on the word “alah” at the end of the passuk: “Alah— tehei meruchak ad sheyavo Moshe shekasuv bo ‘Umoshe alah el ha’Elokim’ v’karevcha v’yomar ‘Yechi Reuven ve’al yamus.” What is the shaychus of Reuven’s kappara to Moshe’s kabbalas haTorah? The answer (from Harav Morgenstern of Beis Medrosh LaTorah) is this:

Chazal say that Hashem said “barasi yetzer hara, barasi Torah tavlin.” Until matan Torah, teshuva removed the pegam of a chet, but did nothing for the underlying middah. Therefore, despite Reuven’s teshuva, the middah megunah remained a middah megunah. But tavlin is a spice, which changes a food’s flavor and makes it taste better. Torah enables a person to change his middah ra’ah and make it into a middah that serves Hashem. After matan Torah, the middah of the shevet of Reuven that prevented them from reaching their potential became responsive to change and improvement, it became a potential tool for avodas Hashem, in the sense of ‘bishtei yetzirecha.’

It’s worth thinking about this— first, teshuva that doesn’t address the underlying problem may be teshuva, but it doesn’t really solve anything, as the Mesillas Yeshorim says. Second, obviously, just learning doesn’t do a thing for a person’s middos ra’os. There has to be an intention and an effort to change through the limud haTorah.