Repository navigation
Conversation
Reviewer's Guideワークスペース登録のメタデータ自動補完を、通常 DOI に加えて arXiv ID・各種 URL・arXiv DOI からも利用できるようにし、arXiv API の未該当・エラー応答で失敗しないよう入力判定、問い合わせ先、応答処理、UI 文言、テストを更新した。 Sequence diagram for arXiv ID metadata lookupsequenceDiagram
actor User
participant UI as WorkspaceRegistrationUI
participant arXivURL
participant ArxivAPI as arXivAPI
participant Utils as MetadataUtils
User->>UI: Enter DOI or arXiv ID
User->>UI: Click metadata retrieval
UI->>arXivURL: __init__(input)
arXivURL->>arXivURL: extract_arxiv_id(input)
alt arXiv ID detected
UI->>arXivURL: _create_url()
arXivURL->>ArxivAPI: GET id_list=arXivID
else Regular DOI
UI->>arXivURL: _create_url()
arXivURL->>ArxivAPI: GET search_query=doi:DOI
end
ArxivAPI-->>Utils: API response
Utils->>Utils: get_arXiv_record_data(input, item_type_id)
alt Valid entry
Utils-->>UI: Metadata result
else No entry or error entry
Utils-->>UI: Empty result
end
Flow diagram for arXiv identifier extractionflowchart LR
Input["DOI, arXiv ID, URL, or arXiv DOI"] --> Extract[extract_arxiv_id]
Extract -->|arXiv ID formats match| ID["Normalized arXiv ID"]
Extract -->|Regular DOI or invalid input| DOI["No arXiv ID"]
ID --> IDURL["id_list API URL"]
DOI --> DOIURL["search_query=doi API URL"]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
 |
🔍 Claude レビュー統合Claude の追加指摘 1 件 — 🔴 高 0 ／ 🟠 中 0 ／ 🟡 低 1
1. 🟡 [低] arXiv ID で取得した論文は arxiv:doi を持たないことが多く、relation の DOI 値が None になる（Claude の追加指摘）
これまで arXiv の検索は DOI 指定だけだったので、ヒットした entry には arxiv:doi が付いていました。今回 id_list 検索を追加したため、ジャーナル DOI が無い preprint も取得できます。その場合 修正案 data_doi が falsy のときは DOI 用の relation 要素を追加しないようにする( 根拠確認: utils.py:2446-2447 と 2538-2565 を読みました。None チェックはありません。build_record_model が None 値をどう扱うかは未確認です。 次にすること: 検証した範囲では修正が必要な問題は見つかりませんでした。arXivURL の呼び出し元は utils.py の get_arXiv_record_data だけで、views.py のエンドポイントはそこから呼ばれており、戻り値の形も変わっていません。ID は正規表現で検証してから id_list に使っているため、URL への入力混入も起きません。既存のレビューは自動生成の要約とスキップ通知だけで、裁定すべき指摘スレッドはありませんでした。 モデル sonnet ／ 2 回実行して和集合 ／ コスト $0.2750。同じ入力でも結果が揺れるため複数回まわし、一部のパスでしか挙がらなかったものには回数を添えています 他レビューを踏まえた自動レビューです。誤りが含まれることがあります。 |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 5 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="modules/weko-workspace/weko_workspace/api.py" line_range="483" />
<code_context>
"""
if not doi:
raise ValueError('DOI is required.')
+ self._arxiv_id = extract_arxiv_id(doi)
self._doi = "/".join(doi.strip().strip("/").split("/")[-2:])
if timeout:
</code_context>
<issue_to_address>
**Cached lookups return wrong metadata**
When two ID/item-type combinations whose concatenated values are identical are requested within the cache timeout, `cached_api_json` builds its cache key by concatenating the ID and item-type ID without a separator, so `get_arXiv_record_data` can return the first lookup's cached record for the other combination and the caller receives incorrect metadata.
Build the cache key from an unambiguous representation of both the ID and item-type ID.
</issue_to_address>
### Comment 2
<location path="modules/weko-workspace/weko_workspace/api.py" line_range="457" />
<code_context>
+ """
+ if not isinstance(value, str):
+ return None
+ value = ARXIV_PREFIX_PATTERN.sub('', value.strip().rstrip('/'))
+ if value.lower().endswith('.pdf'):
+ value = value[:-4]
</code_context>
<issue_to_address>
**Valid arXiv URLs fail autofill**
When a user pastes an arXiv abstract or PDF URL containing a query string or fragment, `extract_arxiv_id` removes the URL prefix but leaves the query string or fragment attached, so `ARXIV_ID_PATTERN` rejects the identifier and the lookup falls back to DOI search without finding arXiv metadata.
Remove the query string or fragment before validating the extracted arXiv identifier.
Also at `modules/weko-workspace/weko_workspace/api.py:438-442`.
</issue_to_address>
### Comment 3
<location path="modules/weko-workspace/weko_workspace/utils.py" line_range="2404-2406" />
<code_context>
+ if isinstance(entry, list):
+ entry = entry[0] if entry else None
+ feed['entry'] = entry
+ if not isinstance(entry, dict) \
+ or '/api/errors' in str(entry.get('id', '')):
+ return result
api_data = get_arXiv_data_by_key(api_response, 'all')
items = ItemTypes.get_by_id(item_type_id)
</code_context>
<issue_to_address>
**API outage appears as no match**
When the arXiv service responds with a non-200 status, `arXivURL.get_data` leaves `response` empty and `error` blank for non-200 statuses, so this guard returns `[]`; the view then reports an ordinary no-result response instead of surfacing the service failure.
Log non-200 responses and set an error value so the view can distinguish service failures from genuine no-match results.
</issue_to_address>
### Comment 4
<location path="modules/weko-workspace/tests/test_utils.py" line_range="3689" />
<code_context>
+ {"feed": {"entry": {"id": "http://arxiv.org/api/errors#incorrect_id_format_for_abc", "title": "Error"}}},
+ {"feed": {"entry": []}},
+ ]:
+ with patch('weko_workspace.utils.arXivURL.get_data', return_value={'response': response, 'error': ''}):
+ with patch('weko_workspace.utils.get_arXiv_data_by_key') as mock_by_key:
+ current_cache.delete('arXiv_data2301.000011')
</code_context>
<issue_to_address>
**Added tests fail Black formatting**
When the module's `black .` formatting check runs, black reformats these newly added long patch and fixture lines, so the module's formatting check reports findings instead of passing.
Reformat the added test expressions with Black.
Also at `modules/weko-workspace/tests/test_utils.py:3697`.
</issue_to_address>
### Comment 5
<location path="modules/weko-workspace/weko_workspace/api.py" line_range="483" />
<code_context>
"""
if not doi:
raise ValueError('DOI is required.')
+ self._arxiv_id = extract_arxiv_id(doi)
self._doi = "/".join(doi.strip().strip("/").split("/")[-2:])
if timeout:
</code_context>
<issue_to_address>
**arXiv IDs remain documented as DOIs**
`arXivURL` now accepts arXiv IDs as well as DOIs, but its constructor documentation still describes only a DOI parameter. Callers relying on that contract can overlook the newly supported input forms.
Update the constructor and `get_arXiv_record_data` docstrings to state that they accept either a DOI or an arXiv identifier/URL.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 3 findings to address first, and if the ID lookup returns incorrect metadata, that metadata could be carried into a workspace record when the user completes registration; the record can be corrected, but reverting will not undo records already created. Reverting does stop future lookups from this feature.
Blocking findings: modules/weko-workspace/weko_workspace/api.py:483, modules/weko-workspace/weko_workspace/api.py:457, modules/weko-workspace/weko_workspace/utils.py:2406
| """ | ||
| if not doi: | ||
| raise ValueError('DOI is required.') | ||
| self._arxiv_id = extract_arxiv_id(doi) |
There was a problem hiding this comment.
🟡 Medium · Cached lookups return wrong metadata
When two ID/item-type combinations whose concatenated values are identical are requested within the cache timeout, cached_api_json builds its cache key by concatenating the ID and item-type ID without a separator, so get_arXiv_record_data can return the first lookup's cached record for the other combination and the caller receives incorrect metadata.
Build the cache key from an unambiguous representation of both the ID and item-type ID.
Prompt for AI agents
In `modules/weko-workspace/weko_workspace/api.py` at line 483:
**Cached lookups return wrong metadata**
When two ID/item-type combinations whose concatenated values are identical are requested within the cache timeout, `cached_api_json` builds its cache key by concatenating the ID and item-type ID without a separator, so `get_arXiv_record_data` can return the first lookup's cached record for the other combination and the caller receives incorrect metadata.
Build the cache key from an unambiguous representation of both the ID and item-type ID.| """ | ||
| if not isinstance(value, str): | ||
| return None | ||
| value = ARXIV_PREFIX_PATTERN.sub('', value.strip().rstrip('/')) |
There was a problem hiding this comment.
🟡 Medium · Valid arXiv URLs fail autofill
When a user pastes an arXiv abstract or PDF URL containing a query string or fragment, extract_arxiv_id removes the URL prefix but leaves the query string or fragment attached, so ARXIV_ID_PATTERN rejects the identifier and the lookup falls back to DOI search without finding arXiv metadata.
Remove the query string or fragment before validating the extracted arXiv identifier.
Also at modules/weko-workspace/weko_workspace/api.py:438-442.
Prompt for AI agents
In `modules/weko-workspace/weko_workspace/api.py` at line 457:
**Valid arXiv URLs fail autofill**
When a user pastes an arXiv abstract or PDF URL containing a query string or fragment, `extract_arxiv_id` removes the URL prefix but leaves the query string or fragment attached, so `ARXIV_ID_PATTERN` rejects the identifier and the lookup falls back to DOI search without finding arXiv metadata.
Remove the query string or fragment before validating the extracted arXiv identifier.
Also at `modules/weko-workspace/weko_workspace/api.py:438-442`.| if not isinstance(entry, dict) \ | ||
| or '/api/errors' in str(entry.get('id', '')): | ||
| return result |
There was a problem hiding this comment.
🟡 Medium · API outage appears as no match
When the arXiv service responds with a non-200 status, arXivURL.get_data leaves response empty and error blank for non-200 statuses, so this guard returns []; the view then reports an ordinary no-result response instead of surfacing the service failure.
Log non-200 responses and set an error value so the view can distinguish service failures from genuine no-match results.
Prompt for AI agents
In `modules/weko-workspace/weko_workspace/utils.py` at lines 2404-2406:
**API outage appears as no match**
When the arXiv service responds with a non-200 status, `arXivURL.get_data` leaves `response` empty and `error` blank for non-200 statuses, so this guard returns `[]`; the view then reports an ordinary no-result response instead of surfacing the service failure.
Log non-200 responses and set an error value so the view can distinguish service failures from genuine no-match results.| {"feed": {"entry": {"id": "http://arxiv.org/api/errors#incorrect_id_format_for_abc", "title": "Error"}}}, | ||
| {"feed": {"entry": []}}, | ||
| ]: | ||
| with patch('weko_workspace.utils.arXivURL.get_data', return_value={'response': response, 'error': ''}): |
There was a problem hiding this comment.
⚪ Low · Added tests fail Black formatting
When the module's black . formatting check runs, black reformats these newly added long patch and fixture lines, so the module's formatting check reports findings instead of passing.
Reformat the added test expressions with Black.
Also at modules/weko-workspace/tests/test_utils.py:3697.
Prompt for AI agents
In `modules/weko-workspace/tests/test_utils.py` at line 3689:
**Added tests fail Black formatting**
When the module's `black .` formatting check runs, black reformats these newly added long patch and fixture lines, so the module's formatting check reports findings instead of passing.
Reformat the added test expressions with Black.
Also at `modules/weko-workspace/tests/test_utils.py:3697`.From AGENTS.md L70-73:
- フォーマッター: **Black** を使用（`black .` でソースコードを整形）
- リンター: **Flake8** を使用（`flake8` で静的解析チェック）
- インポート順の整理: **isort** を使用（`isort .` でインポート並び替え）
- これらのフォーマットチェックはコミット前に必ず実行し、指摘がない状態にしてください| """ | ||
| if not doi: | ||
| raise ValueError('DOI is required.') | ||
| self._arxiv_id = extract_arxiv_id(doi) |
There was a problem hiding this comment.
⚪ Low · arXiv IDs remain documented as DOIs
arXivURL now accepts arXiv IDs as well as DOIs, but its constructor documentation still describes only a DOI parameter. Callers relying on that contract can overlook the newly supported input forms.
Update the constructor and get_arXiv_record_data docstrings to state that they accept either a DOI or an arXiv identifier/URL.
Prompt for AI agents
In `modules/weko-workspace/weko_workspace/api.py` at line 483:
**arXiv IDs remain documented as DOIs**
`arXivURL` now accepts arXiv IDs as well as DOIs, but its constructor documentation still describes only a DOI parameter. Callers relying on that contract can overlook the newly supported input forms.
Update the constructor and `get_arXiv_record_data` docstrings to state that they accept either a DOI or an arXiv identifier/URL.
ワークスペースのアイテム登録画面で、DOI に加えて arXiv ID でも arXiv からメタデータを自動補完できるようにする。 - arXiv ID(新形式 2301.00001 / 旧形式 hep-th/9901001、arXiv: 接頭辞・ バージョン付き)、arxiv.org の abs/pdf URL、arXiv DOI(10.48550/arXiv.) を判定し、arXiv API の id_list で問い合わせる - 通常の DOI は従来どおり search_query=doi: で問い合わせる - arXiv の応答に entry が無い(該当なし)、またはエラー entry の場合は 補完なしとして扱う(従来は KeyError でエラー表示になっていた) - 入力欄のプレースホルダーとアラート文言を更新 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
概要
ワークスペースのアイテム登録画面(
/workspace/item_registration)の DOI 入力欄で、DOI に加えて arXiv ID からも arXiv のメタデータを自動補完できるようにします。目的
これまで arXiv への問い合わせは
search_query=doi:<DOI>だけでした。この検索は arXiv のレコードに登録された出版社 DOI(arxiv:doi)が対象のため、次のケースでは補完できませんでした。2301.00001など)を入力した場合10.48550/arXiv.xxxx)を入力した場合変更内容
api.py:extract_arxiv_id()を追加。arXiv ID と判定した入力はid_list=で問い合わせ、それ以外は従来どおりsearch_query=doi:で問い合わせる2301.00001/ 旧形式hep-th/9901001、arXiv:接頭辞、vN付き、arxiv.org/abs|pdfの URL、10.48550/arXiv.の DOI10.で始まるため、arXiv ID と誤判定しないconfig.py:WEKO_WORKSPACE_ARXIV_ID_API_URLを追加(既存のWEKO_WORKSPACE_ARXIV_API_URLは変更なし)utils.py: arXiv の応答にentryが無い(該当なし)場合とエラー entry の場合は、補完なしとして[]を返す。複数件ヒットしたときは先頭の 1 件を使うKeyErrorになり、「An error have occurred!」と表示されていた動作確認
scripts/ci/run-local.sh weko-workspace: 293 passed, 1 xfailedextract_arxiv_idの判定 23 パターン、id_listの URL 生成、該当なし・エラー entry・空リスト・複数件の扱いid_list=1706.03762→ 1 件ヒットsearch_query=doi:10.48550/arXiv.1706.03762→ 0 件(entryなし)。DOI 検索では arXiv DOI が引けないことと、該当なしの応答形式を確認1706.03762またはhttps://arxiv.org/abs/1706.03762を入力して「取得」を押し、「arXiv MetaData」が選べるようになることを確認する補足
🤖 Generated with Claude Code
Summary by Sourcery
Support arXiv ID-based metadata autofill in workspace item registration while making unsuccessful arXiv lookups return no results safely.
New Features:
Bug Fixes:
Enhancements:
Tests: