I was trying to theme a specific vocabulary and a specific view mode and the suggestions, by default, didn't allow for this. After some searching I came across this article, which mentioned using the hook:
hook_theme_suggestions_HOOK_alter().
So, I added the function below in my .theme file to make the following template suggestion available: 'taxonomy_term__VOCABULARY__VIEW_MODE'
/**
* Implements hook_theme_suggestions_HOOK_alter().
*/
function MYTHEME_theme_suggestions_taxonomy_term_alter(&$suggestions, &$vars) {
// Give a custom template suggestion for the taxonomy type and view mode.
$suggestions[] = $suggestions[0] . "__" . $vars['elements']['#view_mode'];
}