Would it be possible to show the placeholders only when dragging members around, or to hide them completely along with the rest of the frame? Running a sub-optimal party like this seems like a "veteran" choice and shouldn't negatively impact the accessibility of the game at all, if that's your concern.
It would improve the look of 1-3 member parties a lot, methinks.