I mean... Yeah, I agree that skill should be EXPONENTIALLY more important than luck, but sometimes those little bits and pieces add up. "Luck" is just a concept, anyhow; in theory, a group that is absurdly well-prepared and geared to fight should have no trouble with the enemy they're planning to face off against. It's just that they may overlook something that they did not anticipate in being a factor may end up occurring and blame it on bad luck, when in reality that isn't the case.

Though I think we can agree that if dice ARE used, the outliers should be eliminated on the spot unless it fits the narrative better (which should probably be at our GM's discretion). I'm not sure how you'd translate these minute changes over rolls, though, since in situations where luck becomes pivotal a minor slip-up might not change much in the long run... Especially in cases where the RPer is playing a character that might not have the necessary skill to capitalize.

In this regard, I see the dice as a way to tip the scales between equal opponents one way or the other, or a way to give that little hope spot to a group in dire need of aid. It shouldn't be used to indiscriminately make everyone's lives miserable (tbh, they'll probably do that to each other instead), but rather just be used sparingly and only when the situation calls for it. Once the characters have enough experience killing, say, slimes, there shouldn't be a need to roll for those fights because they've learned their opponents in and out and can thus act accordingly.