| 2026-02-11 07:40:09,472 INFO StreamThr :1289371 [internal.py:wandb_internal():85] W&B internal server running at pid: 1289371, started at: 2026-02-11 07:40:09.470139 |
| 2026-02-11 07:40:09,474 DEBUG HandlerThread:1289371 [handler.py:handle_request():158] handle_request: status |
| 2026-02-11 07:40:09,505 INFO WriterThread:1289371 [datastore.py:open_for_write():87] open: /proj/cvl/users/x_fahkh2/WorldModels/WorldMem_Repro/reproduce_official_set/wandb/run-20260211_074009-ku1rnk9p/run-ku1rnk9p.wandb |
| 2026-02-11 07:40:09,508 DEBUG SenderThread:1289371 [sender.py:send():391] send: header |
| 2026-02-11 07:40:09,566 DEBUG SenderThread:1289371 [sender.py:send():391] send: run |
| 2026-02-11 07:40:09,580 INFO SenderThread:1289371 [sender.py:_setup_resume():761] checking resume status for turlin/worldmem/ku1rnk9p |
| 2026-02-11 07:40:10,091 ERROR SenderThread:1289371 [internal_api.py:execute():349] 403 response executing GraphQL. |
| 2026-02-11 07:40:10,091 ERROR SenderThread:1289371 [internal_api.py:execute():350] {"errors":[{"message":"permission denied","path":["upsertBucket"],"extensions":{"code":"PERMISSION_ERROR"}}],"data":{"upsertBucket":null}} |
| 2026-02-11 07:40:10,091 ERROR SenderThread:1289371 [sender.py:send_run():1021] It appears that you do not have permission to access the requested resource. Please reach out to the project owner to grant you access. If you have the correct permissions, verify that there are no issues with your networking setup.(Error 403: Forbidden) |
| Traceback (most recent call last): |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/sdk/lib/retry.py", line 131, in __call__ |
| result = self._call_fn(*args, **kwargs) |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/sdk/internal/internal_api.py", line 345, in execute |
| return self.client.execute(*args, **kwargs) # type: ignore |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/vendor/gql-0.2.0/wandb_gql/client.py", line 52, in execute |
| result = self._get_result(document, *args, **kwargs) |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/vendor/gql-0.2.0/wandb_gql/client.py", line 60, in _get_result |
| return self.transport.execute(document, *args, **kwargs) |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/sdk/lib/gql_request.py", line 59, in execute |
| request.raise_for_status() |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/requests/models.py", line 1026, in raise_for_status |
| raise HTTPError(http_error_msg, response=self) |
| requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.wandb.ai/graphql |
|
|
| During handling of the above exception, another exception occurred: |
|
|
| Traceback (most recent call last): |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/sdk/internal/sender.py", line 1019, in send_run |
| server_run = self._init_run(run, config_value_dict) |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/sdk/internal/sender.py", line 1091, in _init_run |
| server_run, inserted, server_messages = self._api.upsert_run( |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/apis/normalize.py", line 73, in wrapper |
| raise err |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/apis/normalize.py", line 41, in wrapper |
| return func(*args, **kwargs) |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/sdk/internal/internal_api.py", line 2224, in upsert_run |
| response = self.gql( |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/sdk/internal/internal_api.py", line 317, in gql |
| ret = self._retry_gql( |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/sdk/lib/retry.py", line 147, in __call__ |
| retry_timedelta_triggered = check_retry_fn(e) |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/util.py", line 968, in check_retry_fn |
| return fallback_retry_fn(e) |
| File "/proj/cvl/users/x_fahkh2/envs/worldmem/lib/python3.10/site-packages/wandb/util.py", line 910, in no_retry_auth |
| raise CommError( |
| wandb.errors.CommError: It appears that you do not have permission to access the requested resource. Please reach out to the project owner to grant you access. If you have the correct permissions, verify that there are no issues with your networking setup.(Error 403: Forbidden) |
| 2026-02-11 07:40:11,653 DEBUG HandlerThread:1289371 [handler.py:handle_request():158] handle_request: shutdown |
| 2026-02-11 07:40:11,653 INFO HandlerThread:1289371 [handler.py:finish():884] shutting down handler |
| 2026-02-11 07:40:12,114 INFO SenderThread:1289371 [sender.py:finish():1631] shutting down sender |
| 2026-02-11 07:40:12,566 INFO WriterThread:1289371 [datastore.py:close():296] close: /proj/cvl/users/x_fahkh2/WorldModels/WorldMem_Repro/reproduce_official_set/wandb/run-20260211_074009-ku1rnk9p/run-ku1rnk9p.wandb |
|
|
