Wouldn't be the normal notifications good enough?

So we should have both, when out of app they should get push notifications
In-app they dont get push notifications, but do get a little blip with the text of the new comment