SPOILER ALERTS
It wasn't stealing...They were still library books, even if Dumbledore had taken them from the shelves.So right Hermione, and really Dumbledore must have known, as smart as he was, that removing books from the library only would make them that much more attractive, and as Hermione also points out, if Dumbledore hadn't wanted them to have the books he wouldn't have made getting hold of them as easy as using a simple summoning charm!
In London the friends discover, by looking at library records, that the orphanage where Tom Riddle (a.ka. Lord Voldemort) was raised had been demolished. Convinced that the Dark Lord wouldn't have hidden anything as valuable as a piece if his soul in the place he most wanted to escape anyway, the three move on.
Happy Birthday, Harry - the boy who lived! You need no horcruxes, nor hallows, nor sorcerer's stones in order to live forever. Eternal life is yours through the magic of books, as new generations of readers will discover your story.
Always.