XInputGetState/XInputGetCapabilities possibly blocks WMI operation

Discussion:

(too old to reply)

Lin

2007-09-04 16:08:08 UTC

This was a post to microsoft.public.win32.programmer.wmi, but it might be
related to XInput stuff, so forwarded it here as well. (seems WMI group is
not very active)

----------------------------------------------

It seems that XInputGetState call from another thread (invoked once per
second, priority NORMAL) somehow blocks the normal WMI (related) operations.

We found out that by freezing all the suspecious threads of our application,
and then unfreeze them one by one. And the thread calling XInputGetState is
thus spotted.

Commenting out the code calling XInputGetState in our experiments makes WMI
process responsive again. However, XInputGetState has to be invoked regularly
to detect Xbox360 controller connectivity - which even is suggested by MSDN
documentation to be invoked on a framely basis (but calling it once every
second seems fullfill our needs already)

I also noticed that the sample code from the URL
"http://msdn2.microsoft.com/en-US/library/bb173051.aspx" actually utilizes
WMI services, thus I am suspecting that XInputGetState internally actually
relies on WMI doing the real work. Could that potentially interfere with our
own WMI query operations (FYI, what we are querying are Win32_NetworkAdapter
and Win32_NetworkAdapterConfiguration)

Thanks for any helpful comments!

Are there any known issues with the WMI thread possibly being starved by
other application level threads (all in the NORMAL thread priority) in the
same process (in the NORMAL process priority group) which utilizes WMI
service in the semi-async manner? (C++, single core CPU)
The application is realtime DX9 based and graphics intensive, with several
NORMAL priority threads (including the main thread, with minimal waiting in
favor of the application's main looping), and several higher than NORMAL
priority threads for sound mixing (with appropriate blocking at regular
intervals).
We first suspected that the higher than NORMAL threads were starving the WMI
thread (with priority NORMAL), but we found out that if it starved the WMI
thread at thread priority NORMAL, it should also have starved the main
thread, however, it is not true, since the main thread and other threads are
running in their full capacity (renders well, very responsive); and the
higher than NORMAL priority threads all acting very nice to sleep at regular
intervals (that's why they don't starve the main thread)
Now we suspect that it is actually the main thread is starving the WMI
thread; OR the whole application process is actually starving the background
WMI service (in another process) and we believe that the latter might be
more probable.
We then looked at process/thread priority boost when our application is in
the foreground (since our application process is in priority NORMAL process
priority group, as well as the WMI service process, or it's not true?),
things didn't get any better even if we minimized our application.
The WMI stuff seems get responsive at least in 30 seconds interval (we are
basically posting async queries in a continuous manner when the last query
finishes with Indicate/SetStatus callback)
When our application runs in dedicated text mode (without DX9 graphics
intensive stuff), it seems WMI stuff is more responsive, but not at its
extreme; acceptable, however)
How is WMI stuff internally behave? is it behaving too nicely that if other
process (even with the same priority group) is busy one something, it yields
its processing time, and wait indefinitely?
As a side note (might be related?) - we also noticed not only the WMI
service seemed starved, even the normal IO manager seemed somehow starved
that we don't get IO completion notification when we should, when running in
intensive DX9 based system on single core machines. Putting some sleep in
the main thread seemed making the Windows IO manager responsive again (we
get completed IO notifications again)
Thanks!

ZMan

2007-09-05 15:55:31 UTC

Permalink

I've sent this on to the XInput guys... hope to have a response soon..

Post by Lin
This was a post to microsoft.public.win32.programmer.wmi, but it might be
related to XInput stuff, so forwarded it here as well. (seems WMI group is
not very active)
----------------------------------------------
It seems that XInputGetState call from another thread (invoked once per
second, priority NORMAL) somehow blocks the normal WMI (related) operations.
We found out that by freezing all the suspecious threads of our application,
and then unfreeze them one by one. And the thread calling XInputGetState is
thus spotted.
Commenting out the code calling XInputGetState in our experiments makes WMI
process responsive again. However, XInputGetState has to be invoked regularly
to detect Xbox360 controller connectivity - which even is suggested by MSDN
documentation to be invoked on a framely basis (but calling it once every
second seems fullfill our needs already)
I also noticed that the sample code from the URL
"http://msdn2.microsoft.com/en-US/library/bb173051.aspx" actually utilizes
WMI services, thus I am suspecting that XInputGetState internally actually
relies on WMI doing the real work. Could that potentially interfere with our
own WMI query operations (FYI, what we are querying are
Win32_NetworkAdapter
and Win32_NetworkAdapterConfiguration)
Thanks for any helpful comments!

ZMan

2007-09-05 20:57:08 UTC

Permalink

From the XInput team
" The current work-around is to poll for connected devices every frame, but
unconnected devices less frequently (say every two seconds), since that
is reasonable timing (though not ideal) for detecting new devices."

They are also looking into the issue, though mo promises on anything at this
point.

Let us know if this workaround helps your solution.

Post by ZMan
I've sent this on to the XInput guys... hope to have a response soon..

Matthew Coill

2007-09-06 01:20:26 UTC

Permalink

Hi Lin,

The calls for XInputGetState do not use WMI. They use the SetupAPI
(SetupDiEnumDeviceInterfaces etc).

If you could give me some example code, I could try this and figure out what
call in XInput (most likely one of the SetupDi calls) is actually causing
the problem, from there we might be able to suggest a work around.

What is really interesting is that calling XInputGetState once per second is
causing this to block, I could understand if you were calling it once per
frame.

Also, does this happen when querying for a controller when there is none
connected?
XInputGetState(0, ...) with a controller attached to VPort 0? Or only when
there is no controller connected. The code paths are different. If there is
a controller connected there are no SetupApi calls, just CreateFile(...) and
DeviceIoControl(...) calls.

Thanks,
- Matt
The code you pointed out with WMI, is not part of XInput, just sample code
for determining if a DInput device is actually an XInput device.

Lin

2007-09-06 15:50:05 UTC

Permalink

Thanks Matthew and ZMan for the responses.

What we are basically doing is using a dedicated xbox360 controller
(gamepad) connectivity detection thread, whose sole responsibility is to use
XInputGetCapabilities (or XInputGetState in my own experiment) to setup a
lock accessed array of connected gamepads to be polled on a framely basis
from the main thread loop.

The logic of the above mentioned thread is very simple: (pseudo code, since
I don't have the original code at the moment, the actual logic might be
slightly different, and I was not the author of the original logic either)

bool g_connected[4];

void gamepads_connectivity_detection_thread()
{
while (running)
{
Sleep(1000);

{
SCOPED_GLOBAL_LOCK;
for (int i = 0; i < 4; ++i)
{
XINPUT_STATE state;
// commenting out this line makes WMI process (wmiprvse.exe)
responsive again
g_connected[i] = XInputGetCapabilities(i,
XINPUT_FLAG_GAMEPAD, &caps) == ERROR_SUCCESS);
}
}
}
}

void main()
{
while (running)
{
std::vector<XINPUT_STATE> states;

{
SCOPED_GLOBAL_LOCK;
for (int i = 0; i < 4; ++i)
{
if (g_connected[i])
{
XINPUT_STATE state;
if (XInputGetState(i, &state) == ERROR_SUCCESS)
states.push_back(state);
}
}
}

for (size_t i = 0; i < states.size(); ++i)
ProcessXInputState(states[i]);
}
}

The code for querying WMI for Win32_NetworkAdapter and Win32_NetworkAdapter
configuration is not given here due to its complextiy and also I don't have
the code at the moment either (but yes, I wrote the stuff and tested it in a
seperate environment without any XInput invocation, it worked just fine, as
well as in our application with the XInputGetCapabilities call commented out.)

I'll get back to you guys when I return to work next week after making
(even) more extensive tests.

Lin

FYI, I have done some tests at home (without the original source code at
hand and environment setup), but seems everything works FINE with the tests
below.

very simple C++ XInput test program:

int _tmain(int argc, _TCHAR* argv[])
{
while (true)
{
for (DWORD i = 0; i < 4; ++i)
{
//XINPUT_STATE state;
//ZeroMemory(&state, sizeof(state));
//DWORD r = XInputGetState(i, &state);
XINPUT_CAPABILITIES caps;
ZeroMemory(&caps, sizeof(caps));
DWORD r = XInputGetCapabilities(i, XINPUT_FLAG_GAMEPAD, &caps);
printf("r=%u\n", r);
}

//Sleep(1000);
}

return 0;
}

very simple C# WMI test program (note that it is synchronous)

using System;
using System.Management;

namespace WMITest
{
class clsMain
{
static void Main(string[] args)
{
while (true)
GetWMIStats();
}

static void GetWMIStats()
{
ConnectionOptions oConn = new ConnectionOptions();
System.Management.ManagementScope oMs = new
System.Management.ManagementScope("\\\\localhost", oConn);

System.Management.ObjectQuery q1 = new
System.Management.ObjectQuery("select * from Win32_NetworkAdapter");
System.Management.ObjectQuery q2 = new
System.Management.ObjectQuery("select * from
Win32_NetworkAdapterConfiguration");
ManagementObjectSearcher s1 = new ManagementObjectSearcher(oMs,
q1);
ManagementObjectSearcher s2 = new ManagementObjectSearcher(oMs,
q2);
ManagementObjectCollection c1 = s1.Get();
ManagementObjectCollection c2 = s2.Get();

foreach (ManagementObject r1 in c1)
Console.WriteLine(r1.ToString());
foreach (ManagementObject r2 in c2)
Console.WriteLine(r2.ToString());
}
}
}

Matthew Coill

2007-09-07 19:24:38 UTC

Permalink

Thanks Lin,

I will try running this code to see if I get the same blockage, if not can I
send you code that does the same exact thing as XInputGetState(...) and have
you track down which call is causing the blockage?

Also curious, if you change the thread to just search for one joystick, does
it make a difference if the joystick is connected?

Thanks,
- Matt

Post by Lin
Thanks Matthew and ZMan for the responses.
What we are basically doing is using a dedicated xbox360 controller
(gamepad) connectivity detection thread, whose sole responsibility is to use
XInputGetCapabilities (or XInputGetState in my own experiment) to setup a
lock accessed array of connected gamepads to be polled on a framely basis
from the main thread loop.
The logic of the above mentioned thread is very simple: (pseudo code, since
I don't have the original code at the moment, the actual logic might be
slightly different, and I was not the author of the original logic either)
bool g_connected[4];
void gamepads_connectivity_detection_thread()
{
while (running)
{
Sleep(1000);
{
SCOPED_GLOBAL_LOCK;
for (int i = 0; i < 4; ++i)
{
XINPUT_STATE state;
// commenting out this line makes WMI process
(wmiprvse.exe)
responsive again
g_connected[i] = XInputGetCapabilities(i,
XINPUT_FLAG_GAMEPAD, &caps) == ERROR_SUCCESS);
}
}
}
}
void main()
{
while (running)
{
std::vector<XINPUT_STATE> states;
{
SCOPED_GLOBAL_LOCK;
for (int i = 0; i < 4; ++i)
{
if (g_connected[i])
{
XINPUT_STATE state;
if (XInputGetState(i, &state) == ERROR_SUCCESS)
states.push_back(state);
}
}
}
for (size_t i = 0; i < states.size(); ++i)
ProcessXInputState(states[i]);
}
}
The code for querying WMI for Win32_NetworkAdapter and
Win32_NetworkAdapter
configuration is not given here due to its complextiy and also I don't have
the code at the moment either (but yes, I wrote the stuff and tested it in a
seperate environment without any XInput invocation, it worked just fine, as
well as in our application with the XInputGetCapabilities call commented out.)
I'll get back to you guys when I return to work next week after making
(even) more extensive tests.
Lin
FYI, I have done some tests at home (without the original source code at
hand and environment setup), but seems everything works FINE with the tests
below.
int _tmain(int argc, _TCHAR* argv[])
{
while (true)
{
for (DWORD i = 0; i < 4; ++i)
{
//XINPUT_STATE state;
//ZeroMemory(&state, sizeof(state));
//DWORD r = XInputGetState(i, &state);
XINPUT_CAPABILITIES caps;
ZeroMemory(&caps, sizeof(caps));
DWORD r = XInputGetCapabilities(i, XINPUT_FLAG_GAMEPAD, &caps);
printf("r=%u\n", r);
}
//Sleep(1000);
}
return 0;
}
very simple C# WMI test program (note that it is synchronous)
using System;
using System.Management;
namespace WMITest
{
class clsMain
{
static void Main(string[] args)
{
while (true)
GetWMIStats();
}
static void GetWMIStats()
{
ConnectionOptions oConn = new ConnectionOptions();
System.Management.ManagementScope oMs = new
System.Management.ManagementScope("\\\\localhost", oConn);
System.Management.ObjectQuery q1 = new
System.Management.ObjectQuery("select * from Win32_NetworkAdapter");
System.Management.ObjectQuery q2 = new
System.Management.ObjectQuery("select * from
Win32_NetworkAdapterConfiguration");
ManagementObjectSearcher s1 = new ManagementObjectSearcher(oMs,
q1);
ManagementObjectSearcher s2 = new ManagementObjectSearcher(oMs,
q2);
ManagementObjectCollection c1 = s1.Get();
ManagementObjectCollection c2 = s2.Get();
foreach (ManagementObject r1 in c1)
Console.WriteLine(r1.ToString());
foreach (ManagementObject r2 in c2)
Console.WriteLine(r2.ToString());
}
}
}

Lin

2007-09-07 20:46:01 UTC

Permalink

Thanks Matthew,

It would be great if you could send me code that does the same exact thing
as XInputGetState function and let me try tracking down which call is
possibly causing the blockage (if you don't have any luck reproducing it
there :).

When I saw the issue occuring, one xbox360 controller was always connected.
I will make tests (connecting and unconnecting the gamepad, and poll for the
controller only at slot 0, etc.) as you suggested first thing next Monday.

FYI only, my workstation was with a single core AMD 64 CPU, it had the
problem, and then changed to a dual core AMD 64 CPU, and I have the same
problem still. But the test code I pasted at the end of my last post was run
on my home computer with a hyperthreaded Intel CPU, and it seemed that I was
not able to reproduce the problem (but it might also be how the test programs
were written - they are pretty different from the real world code).

If you would send me any code that is not appropriate for public domain, my
email address at work is: lin [at] crytek [dot] de

I will be keeping communicating with you through direct email when I get
back to work the next week.

Thank you again for your help!

Post by Matthew Coill
Thanks Lin,
I will try running this code to see if I get the same blockage, if not can I
send you code that does the same exact thing as XInputGetState(...) and have
you track down which call is causing the blockage?
Also curious, if you change the thread to just search for one joystick, does
it make a difference if the joystick is connected?
Thanks,
- Matt

Matthew Coill

2007-09-07 21:20:36 UTC

Permalink

Taking off-line

Thanks,
- Matt

Post by Lin
Thanks Matthew,
It would be great if you could send me code that does the same exact thing
as XInputGetState function and let me try tracking down which call is
possibly causing the blockage (if you don't have any luck reproducing it
there :).
When I saw the issue occuring, one xbox360 controller was always connected.
I will make tests (connecting and unconnecting the gamepad, and poll for the
controller only at slot 0, etc.) as you suggested first thing next Monday.
FYI only, my workstation was with a single core AMD 64 CPU, it had the
problem, and then changed to a dual core AMD 64 CPU, and I have the same
problem still. But the test code I pasted at the end of my last post was run
on my home computer with a hyperthreaded Intel CPU, and it seemed that I was
not able to reproduce the problem (but it might also be how the test programs
were written - they are pretty different from the real world code).
If you would send me any code that is not appropriate for public domain, my
email address at work is: lin [at] crytek [dot] de
I will be keeping communicating with you through direct email when I get
back to work the next week.
Thank you again for your help!

Post by Lin
Thanks Matthew and ZMan for the responses.
What we are basically doing is using a dedicated xbox360 controller
(gamepad) connectivity detection thread, whose sole responsibility is
to
use
XInputGetCapabilities (or XInputGetState in my own experiment) to setup a
lock accessed array of connected gamepads to be polled on a framely basis
from the main thread loop.
The logic of the above mentioned thread is very simple: (pseudo code, since
I don't have the original code at the moment, the actual logic might be
slightly different, and I was not the author of the original logic either)
bool g_connected[4];
void gamepads_connectivity_detection_thread()
{
while (running)
{
Sleep(1000);
{
SCOPED_GLOBAL_LOCK;
for (int i = 0; i < 4; ++i)
{
XINPUT_STATE state;
// commenting out this line makes WMI process (wmiprvse.exe)
responsive again
g_connected[i] = XInputGetCapabilities(i,
XINPUT_FLAG_GAMEPAD, &caps) == ERROR_SUCCESS);
}
}
}
}
void main()
{
while (running)
{
std::vector<XINPUT_STATE> states;
{
SCOPED_GLOBAL_LOCK;
for (int i = 0; i < 4; ++i)
{
if (g_connected[i])
{
XINPUT_STATE state;
if (XInputGetState(i, &state) == ERROR_SUCCESS)
states.push_back(state);
}
}
}
for (size_t i = 0; i < states.size(); ++i)
ProcessXInputState(states[i]);
}
}
The code for querying WMI for Win32_NetworkAdapter and
Win32_NetworkAdapter
configuration is not given here due to its complextiy and also I don't have
the code at the moment either (but yes, I wrote the stuff and tested it
in
a
seperate environment without any XInput invocation, it worked just
fine,
as
well as in our application with the XInputGetCapabilities call
commented
out.)
I'll get back to you guys when I return to work next week after making
(even) more extensive tests.
Lin
FYI, I have done some tests at home (without the original source code at
hand and environment setup), but seems everything works FINE with the tests
below.
int _tmain(int argc, _TCHAR* argv[])
{
while (true)
{
for (DWORD i = 0; i < 4; ++i)
{
//XINPUT_STATE state;
//ZeroMemory(&state, sizeof(state));
//DWORD r = XInputGetState(i, &state);
XINPUT_CAPABILITIES caps;
ZeroMemory(&caps, sizeof(caps));
DWORD r = XInputGetCapabilities(i, XINPUT_FLAG_GAMEPAD, &caps);
printf("r=%u\n", r);
}
//Sleep(1000);
}
return 0;
}
very simple C# WMI test program (note that it is synchronous)
using System;
using System.Management;
namespace WMITest
{
class clsMain
{
static void Main(string[] args)
{
while (true)
GetWMIStats();
}
static void GetWMIStats()
{
ConnectionOptions oConn = new ConnectionOptions();
System.Management.ManagementScope oMs = new
System.Management.ManagementScope("\\\\localhost", oConn);
System.Management.ObjectQuery q1 = new
System.Management.ObjectQuery("select * from Win32_NetworkAdapter");
System.Management.ObjectQuery q2 = new
System.Management.ObjectQuery("select * from
Win32_NetworkAdapterConfiguration");
ManagementObjectSearcher s1 = new
ManagementObjectSearcher(oMs,
q1);
ManagementObjectSearcher s2 = new
ManagementObjectSearcher(oMs,
q2);
ManagementObjectCollection c1 = s1.Get();
ManagementObjectCollection c2 = s2.Get();
foreach (ManagementObject r1 in c1)
Console.WriteLine(r1.ToString());
foreach (ManagementObject r2 in c2)
Console.WriteLine(r2.ToString());
}
}
}

Lin

2007-09-17 12:20:02 UTC

Permalink

Follow-up:

1. the background Windows processes (like wmiprvse.exe) could be
blocked/starved on Windows based systems that the Windows processor
scheduling is configured in favor of foreground processes than background
processes. More information can be found in the following link:
http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/regentry/29623.mspx?mfr=true

2. XInput is found NOT guilty at all in blocking the background WMI process.
We found that it is one of our own threads that becomes busy looping when one
(or more) XInput controller is connected.

3. it is found out that the starvation of our IOCP bound worker thread is
due to a bit misunderstanding of how IOCP behaves thus making wrong
assumption, causing worker thread blockage on the IOCP due to concurrency
value exceeding the maximum. The Windows IO manager is working correctly

Post by Lin
This was a post to microsoft.public.win32.programmer.wmi, but it might be
related to XInput stuff, so forwarded it here as well. (seems WMI group is
not very active)
----------------------------------------------
It seems that XInputGetState call from another thread (invoked once per
second, priority NORMAL) somehow blocks the normal WMI (related) operations.
We found out that by freezing all the suspecious threads of our application,
and then unfreeze them one by one. And the thread calling XInputGetState is
thus spotted.
Commenting out the code calling XInputGetState in our experiments makes WMI
process responsive again. However, XInputGetState has to be invoked regularly
to detect Xbox360 controller connectivity - which even is suggested by MSDN
documentation to be invoked on a framely basis (but calling it once every
second seems fullfill our needs already)
I also noticed that the sample code from the URL
"http://msdn2.microsoft.com/en-US/library/bb173051.aspx" actually utilizes
WMI services, thus I am suspecting that XInputGetState internally actually
relies on WMI doing the real work. Could that potentially interfere with our
own WMI query operations (FYI, what we are querying are Win32_NetworkAdapter
and Win32_NetworkAdapterConfiguration)
Thanks for any helpful comments!