Some sort of puss/bloody liquid builds up underneath my scab and gives a kind of balloon feel?

See a doctor. I think you have an infectious process going on, especially if there is pus. Your doctor can drain out the infection and cover the wound with sterile bandages. The more you handle the sore, the more bacteria are introduced into it, so don't be a do-it-yourself doc unless you have a degree.