This issue in fact bugs me for months, and it only happens on my personal laptop (a Windows 10). It won't go away even I have cleared the cache for Skype for Business, delete the sign in information, etc.
Until I checked the tracing log file, and it mentioned that there is a loop while accessing my Wi-Fi adapter, I start to narrow down the possible root cause: Network adapter.
And finally I managed to get it fixed, by turning off Internet Protocol Version 6 (IPv6).
