I find this odd because in my experience with KDE Plasma in recent years, both versions 5 and 6, when there are two active connections, one which is wired and one which is wireless, the icon of the Network Connections applet usually defaults to the wired network icon in the System Tray. The wired connection icon usually takes precedence over the wireless icon when both types of connections are active and present. For example, when the wired connection is disabled, disconnected, or disappears, the Network Connections icon in the System Tray changes to a wireless connection icon, to match the only active connection available. When the wired connection is reactivated, the icon changes back to a wired connection icon.
Those icons in the System Tray are taken from the user's current icon theme. When you change the icon theme, does the problem still occur? A stale cache can create icon and theming issues in KDE, especially after updates or changes to themes including icons and fonts.
Icon themes can be cached in ~/.cache. You can try removing all the files in this directory from a TTY when not logged in (they will regenerate after logging back in):
Does this occur with a new user? It might be worth the time and effort it takes to create a new user and see if this behavior still occurs with the stock KDE configuration. If it does not, it is a problem confined to your user's KDE configuration in the home directory. This step would help you determine that and eliminate other causes.
Network Manager by default has a network connectivity check enabled on each connection. This check can fail when using a network tunnel, proxy or a VPN. It could theoretically happen when other non-standard network connections are enabled such as a dummy connection enabled. I have disabled it for this reason on my machines. When the check fails, the icon could possibly exhibit the behavior you describe. I seem to vaguely recall this happening with my systems when using a VPN.
Create the file /etc/NetworkManager/conf.d/20-connectivity.conf with the following contents:
You can restart NetworkManager and network services but the easiest thing to do here would be to restart the computer.
(This works on systemd, I assume it also works on the alternatives. )
For clarity, the Wireless Network icon with the question mark that is being displayed is named network-wireless-acquiring.svg and network-wirless-available.svg from the breeze icon theme, if that gives you any indicators of why Network Manager might be using those icons instead of something more desirable.
In the end, this may be due to having non-standard network connections or network setup. You might just have to make peace with it in that case.