Since this question is getting a lot of attraction, I decided to post what worked for me
Remove Specific User's Specific Reaction
// Channel = the channel object of the message's original channel
// MessageID = ID of the message, if hard coding, put around quotations eg: "1234"
const msg = await channel.messages.fetch(MessageID);
REACTION OBJECT OR REACTION ID").users.remove("ID OR OBJECT OF USER TO REMOVE");
Note: If using an older version of discord.js, simply replace channel.messages.fetch with channel.fetchMessage
discord.js remove reaction