Quota is not reset after waiting RateLimit-Reset time

[digiofficerobin]

Target SharePoint environment

SharePoint Online

What SharePoint development model, framework, SDK or API is this about?

SharePoint CSOM

Developer environment

Windows

What browser(s) / client(s) have you tested

• 💥 Internet Explorer
• 💥 Microsoft Edge
• 💥 Google Chrome
• 💥 FireFox
• 💥 Safari
• mobile (iOS/iPadOS)
• mobile (Android)
• not applicable
• other (enter in the "Additional environment details" area below)

Additional environment details

• Microsoft.SharePointOnline.CSOM 16.1.22810.12000
• .NET 6

Describe the bug / error

While developing and testing the RateLimit headers which SharePoint online provides we noticed that the RateLimit-Reset value does not imply the number of seconds an app should wait before the quota is fully reset.

In the documentation https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online is stated that RateLimit-Reset value is the number of seconds before the quota is reset.

In our testing we see that after we wait the number of seconds specified by the RateLimit-Reset value the quota is not reset. In this example the quota is reset after 26 seconds instead of 5 seconds.

Steps to reproduce

1. Use the example provided by https://github.com/OneDrive/samples/blob/master/scenarios/throttling-ratelimit-handling/readme.md
2. Modify the following lines InitializeAndRunAsync() method in Demo.cs:

From:
parallelOps.Add(DoWork(j, client, sharePointAccessToken, graphAccessToken, 5000));

To:

                if (j == 0)
                {
                    parallelOps.Add(DoWork(j, client, sharePointAccessToken, graphAccessToken, 5000));
                }
                else
                {
		  parallelOps.Add(DoWork(j, client, sharePointAccessToken, graphAccessToken, 90));
		}

4. Use this RateLimiter.cs file (uploaded as txt file)
   RateLimiter.cs.txt

5. In Demo.cs we have changed the code of SharePointCSOMCallAsync() method to make a EnsureUser call to SharePoint online.

        private async Task SharePointCSOMCallAsync(Microsoft.SharePoint.Client.ClientContext clientContext)
        {
            // Make a CSOM call
            var refUserAsPrincipal = clientContext.Web.EnsureUser("ADSynctest-01"); // 2 punten
			clientContext.Load(refUserAsPrincipal, _user => _user.Email, _user => _user.LoginName);

			await clientContext.ExecuteQueryAsync();
			//clientContext.Load(clientContext.Web);
			//await clientContext.ExecuteQueryAsync();
			IncrementCallsDone();

            //Console.WriteLine(clientContext.Web.Title);
        }

6. In Demo.cs we commented out the following lines:

                // Test a SharePoint Rest call
               // await SharePointRestCallAsync(client, sharePointAccessToken);

                // Test a Graph Rest call
                //await GraphRestCallAsync(client, graphAccessToken);

7. Run the example

Expected behavior

We expect that after waiting the RateLimit-Reset value in seconds the quota is reset and the folllow up requests can go full throttle.

[digiofficerobin]

Any updates on this issue?

[Ashlesha-MSFT]

Hello @digiofficerobin,
Thank you for bringing this issue to our attention. We will look into it and get back to you shortly.

[Ashlesha-MSFT]

@digiofficerobin
Thank you for detailed steps but i am getting below error
Can you please help here to resolve error.

[digiofficerobin]

@Ashlesha-MSFT I've added the complete Sample code as an attachment:

ThrottlingSample.zip

There are two similar projects in the folders:

1. ROPC, which uses the Resource Owner Password Credentials auth flow, thus with a service account with username/password (without MFA enabled)
2. ClientCredentialsWithCertificate, which uses the Client Credentials auth flow with a certificate. A certificate is required to make calls with CSOM.

In both projects there is a simple EnsureUser call and you can uncomment the code to load site information which is a much bigger call.

In my testing today I've noticed that both projects do not return RateLimit headers at all at the moment. After a while I get 429 Too many requests responses.

In Program.cs you can change the number of threads that must be spin off.

Please feel free to ask any questions you may have.

[Ashlesha-MSFT]

@digiofficerobin.
Thanks for providing .zip.
Since our tenant uses MFA, we can't use ROPC
For another way to authenticate i.e. through Certificate I am getting below issue
: 'Certificate could not be loaded using this path information My|CurrentUser|<>

since i able to see it has private key and also performed other steps to resolve the issue
HasPrivateKey ------------- True

Could you please help.

[digiofficerobin]

@Ashlesha-MSFT

ROPC flow. You can turn off MFA requirement for specific account. https://learn.microsoft.com/en-us/answers/questions/2079245/how-to-remove-mfa-2fa-requirement-for-one-user-i-c

Credentials flow with certificate. I've followed these steps which worked for me:

1. Create Self Signed certificate https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-self-signed-certificate

Powershell command:

$cert = New-SelfSignedCertificate -Subject "CN=$certname" -CertStoreLocation "Cert:\LocalMachine\My" -KeyExportPolicy Exportable -KeySpec Signature -KeyLength 2048 -KeyAlgorithm RSA -HashAlgorithm SHA256

2. Export public certificate

Export-Certificate -Cert $cert -FilePath "d:$public-certname.cer"

3. Export private certificate

$mypwd = ConvertTo-SecureString -String "ThisIsMyPassword" -Force -AsPlainText
Export-PfxCertificate -Cert $cert -FilePath "d:$private-certname.cer" -Password $mypwd

4. Upload public certificate to Azure app registration
5. Use thumbprint in your app

If you install certificate in LocalMachine store then you must run Visual Studio As Administrator.
Make sure you have access to the certificate. Start certlm > Right-Click on the certificate in certificate manager > All tasks > Manage private keys... > Add your account.

[Ashlesha-MSFT]

Hello @digiofficerobin,
Thanks for help to authenticate
I am able to get rid of certificate authentication issue, but still authentication issue persists in other context
Getting below error
Microsoft.SharePoint.Client.ServerUnauthorizedAccessException: 'Attempted to perform an unauthorized operation.'

I have given additional permissions too

[digiofficerobin]

@Ashlesha-MSFT Do you use the ROPC project or the ClientCredentials project?

With ROPC you'll also need to make the user a SharePoint administrator member in the site collection (Site collection (root) > Site Settings > Site administrators).

My ClientCredentials flow app permissions look like this:

Can you show the callstack so I can see at which request the auth is failing?

[digiofficerobin]

For everybody who uses the RateLimit example of @jansenbe as a starting point, notice that there is a bug in the reporting of the amount of seconds. I've reported this as issue here: https://github.com/OneDrive/samples/tree/master/scenarios/throttling-ratelimit-handling

[Ashlesha-MSFT]

@digiofficerobin,
I am unable to use the Resource Owner Password Credentials (ROPC) flow because turning off Multi-Factor Authentication (MFA) is not an option. This is due to the fact that MFA is required for other users in the organization and disabling it would create dependencies that affect their access and security.

Getting below error using certificate