--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:46935 - loss:MatryoshkaLoss - loss:GuideGISTEmbedLoss widget: - source_sentence: multi-modal imputation computational cost sentences: - 'title: Learning Flexible Forward Trajectories for Masked Molecular Diffusion summary: This paper focuses on the adaptation problem of Masked Diffusion Models (MDMs) for discrete data in molecular graph generation. It points out that "fixed, element-independent" forward masking scheduling leads to different molecules collapsing to the same intermediate state in the forward trajectory, making reverse denoising, typically unimodal and predicting independently by node or edge, difficult to learn the correct reconstruction target. To address this, the paper proposes MELD: which learns the forward masking rate at the element level (node/edge) and assigns an independent erosion trajectory to each graph element through a parameterized noise scheduling network; it is jointly optimized with the reverse denoising network during training. The authors claim that MELD achieves high efficiency in unconditional generation of QM9 and ZINC250K graphs and outperforms standard MDM and several diffusion baselines in distribution alignment and property alignment. weaknesses and questions: 1.The element-level kernel renders the forward process non-equivariant, meaning the intermediate state distribution is affected by vertex permutations. For molecular graphs, this contradicts the fundamental principle that isomorphism should not alter the generative distribution. Current methods merely introduce a learnable embedding H for each graph element and "randomly permutate columns" to "distinguish graph states with the same number of nodes/edges," but this does not restore the guarantee of permutation equivariance. It needs to be proven that this forward process, which breaks equivariance, does not induce dependencies on node labels and generalization issues, especially whether relabeling input nodes during testing maintains a consistent sampling distribution. 2.The abstract and main text claim that MELD is "the first diffusion model to achieve 100% chemigenicity in unconditional generation on QM9 and ZINC250K," but several MDM baselines in Table 1 also show 100%. The wording needs to be corrected. 3.The paper does not provide an explicit collision risk function or upper and lower bound analysis; the loss in Equation (3) does not directly minimize the "collision probability". It is suggested to provide a computable proxy metric and its relationship with the gradient direction, or to supplement the appendix with a simplified derivation of the "collision probability as a function of {𝑤_{𝑖}}". 4.The manuscript states that "unless otherwise specified, standard MDM and MELD use the same DiT backbone," but were the other discrete/continuous diffusion baselines in Table 1 also retrained and had their backbones and training budgets aligned? If comparisons are only made within the MDM family without aligning the backbones/hyperparameters of external distributed models, the conclusions may overestimate the advantages of MELD. Please provide the number of training epochs, GPU configuration, total duration, and FLOPs in the appendix, as well as the retraining/reproduction practices for each baseline. 5.The use of V.U.N.↑ in Tables 3 and 6 lacks a clear explanation of its meaning and calculation in the text (it seems to be a composite score for Validity/Uniqueness/Novelty?). Please define it explicitly at its first appearance in the text. 1.The statement "first 100% validity" conflicts with Table 1. It is recommended to change it to "significantly reduced FCD/NSPDK while maintaining 100% validity." Could you please report the confidence intervals for inefficiency (multiple sampling)? 2.Can a more systematic comparison be made between the key differences and complexity of existing "adaptive/category-level" scheduling (such as DiffusionBERT, GenMD4, TabDiff) and the "element-level" scheduling in this paper? Currently, only a rough comparison is made in Table 3, lacking a theoretical analysis of the differences in expressive power. 3.Please list the number of training epochs, learning rate, scheduler, backbone, number of GPUs, and training time for all baselines; and specify which baselines were retrained by the authors and which were reproduced from the original paper. 4.Please add "Node relabeling robustness test" (variance of distribution index/property MAE under multiple labels of the same molecule). 5.Table 5 only performs isomorphism counting on 12 nodes/131 samples, which is costly but has a small sample size; it is recommended to provide estimation methods for larger scales (such as approximate GI or fingerprint hash upper/lower bounds) and statistical confidence intervals.' - "title: ICFI: A Feature Importance Measure For Multi-Class Classification\n\n\ summary: The paper proposes a novel feature importance method for multi-class\ \ classification. The method tackles the problem of providing not only one set\ \ of feature importances, but one for each pair of classes, thus offering more\ \ insight into the classifier. The method is based on the idea of observing decrease\ \ in empirical risk when two classes are combined, in combination with permutation-based\ \ feature importance. Some experiments are provided showing that the method gives\ \ sensible results and outperforms GSHAP adapted to the same problem.\n\nweaknesses\ \ and questions: The proposed method is relatively simple and does not bring any\ \ extremely innovative methodology or theoretical results, which is nothing wrong\ \ by itself, but the I would expect a very strong empirical evaluation or (even\ \ better) a practical use-case that demonstrates not only that the method works\ \ but that the problem of requiring additional insights into (pairwise) relationships\ \ between classes is really a problem in need of a solution.\n\nThe current experiments\ \ do not convince me (see Questions). As the authors also say, evaluation of\ \ XAI is a big challenge and there doesn't seem to be any shortcut to a sound\ \ empirical evaluation (https://icml.cc/virtual/2025/poster/40169). The first\ \ two experiments establish that there is nothing clearly wrong with the method,\ \ which is OK. The retraining experiment and comparison with GSHAP I do not understand.\ \ If the goal of the method is to provide insights into how the model classifies,\ \ then this is far from a realistic assessment (yes, it is common to do this in\ \ XAI/ML papers, but it doesn't make it any less unrealistic). Also, it seems\ \ to me that GSHAP was forced into this comparison, not being a method developed\ \ for the same purpose. I might be wrong, but the paper doesn't do a good job\ \ of describing exactly what GSHAP is or how it was adapted.\n\nAnd I might have\ \ other issues with the paper on things that I currently don't quite understand\ \ and/or were not explained clearly enough (also see Questions).\n\nMinor comments:\n\ - Some extra effort seems to have gone into squeezing this to fit the page limit\ \ (Figure 6 caption has no space to breathe, etc.).\n- ).One\n- The proposed method\ \ operates on model risk not on model predictions directly. So, technically, it\ \ is not explaining what the model does, but what features contribute to the models\ \ predictive performance. Often the same, but not always.\nQ1: Finally, why not\ \ include some global feature importance into the comparison? The problem of masking\ \ the feature importance of globally less important features that are important\ \ for certain pairs of classes might be exaggerated. I'd imagine that for a low\ \ number of classes the global ordering would be decent (definitely better than\ \ random). \n\nQ2. Computational complexity: First, it would really help if the\ \ computational complexity is stated more explicitly, instead of \"in line with\ \ existing permutation methods but cheaper than SHAP\". Second, I'm not convinced\ \ that the latter is correct. The proposed method requires for each feature a\ \ constant number of permutations and each permutation requires a model prediction?\ \ Any decent implementation of SHAP should also be linear in the number of features\ \ and will contain the model prediction (you don't go through all subsets of coalitions).\n\ \nQ3. Permutation importance has certain failure cases, compared to SHAP, for\ \ example. Why not combine the idea of combining two classes but then use Shapley\ \ values instead of permutation importance? \n\nQ4. I'd remove the explicit \"\ Definition 1\" from definition of the pairwise feature importance problem. It\ \ is not necessary and it is not precise. Informally we would probably agree on\ \ what \"as it pertains to separating the target classes $\\sigma$ and $\\rho$\"\ \ means, but what does it really mean? A model never trully 100% focuses on separating\ \ only two classes (unless there are only two classes).\n\nQ5. The interval computation\ \ in A.3 seems like overkill. The (Bayesian posterior) mean and standard deviation\ \ of a process where 100 independent samples are given is estimated using Markov\ \ Chain Monte Carlo? Unless I'm missing something, the only possible justification\ \ would be that we use uniform priors on the two parameters and therefore can't\ \ use the analytical solution. But if we are going to be so precise as to not\ \ allow values outside of [0,1] then why use a Gaussian likelihood, which is clearly\ \ not appropriate. Burn-in also doesn't make sense (why not just pick a sensible\ \ starting value, like the empirical mean and standard deviation). To summarize,\ \ average +/- 1.96 * standard deviation of the sample / sqrt(100) should give\ \ essentially the same results.\n\nQ6. I'm unsure about the upper/lower bound\ \ requirement. First, the requirements, as stated, would allow for a method that\ \ assigns arbitrarily low negative feature importances (we require irrelevant\ \ features to have 0 and to have an upper bound; there is nothing saying that\ \ a relevant feature can't have a negative importance, for example, if it decreases\ \ predictive performance). I'll assume that the intention was for them to be bounded\ \ between 0 and an upper bound (which might as well be 1). I'm not convinced by\ \ the argument that people prefer bounded things therefore bounding is better.\ \ That is, it is mathematically easy to bound things, but with it we change the\ \ scale of the feature importance. Are these importances even comparable across\ \ class pairs for same risk? Are they comparable across different risks?" - 'title: Learning Optimal Multimodal Information Bottleneck Representations summary: The author introduces a theoretically guaranteed multimodal information bottleneck approach. This method dynamically adjusts the regularization weights of each modality by considering the varying degrees of task-relevant information across different modalities. Theoretically, the optimization objective proposed by the author is of a remarkably straightforward form, and the practical loss function serves as an upper bound to this theoretical objective, thereby ensuring the feasibility. #### update after rebuttal: I don''t change my assessment. weaknesses and questions: The article is logically structured, with clearly defined and reader-friendly symbols. The theoretical section is particularly detailed and rigorous. In the experiments, the non-MIB-based methods only include some basic approaches. I am curious about how they compare with newer methods: Peng, Xiaokang, et al. "Balanced multimodal learning via on-the-fly gradient modulation." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. Zhang, Xiaohui, et al. "Multimodal representation learning by alternating unimodal adaptation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. I harbor a degree of skepticism towards the reasoning in Equation (3), which posits that concatenating $e_i$ enhances the model''s learning by improving the signal-to-noise ratio. Typically, the introduction of signal-to-noise ratio considerations involves additive noise, such as ( z_i^{noise} = z_i + e_i ), rather than direct concatenation. Moreover, the ablation studies do not include relevant content to substantiate this.' - "title: A Structured Pruning Algorithm for Model-based Deep Learning\n\nsummary:\ \ This paper proposes a structured pruning method for model-based deep learning\ \ in inverse problems. The proposed method, SPADE, reduces the computational complexity\ \ of model-based networks at test-time by pruning its non-essential weights. In\ \ addition, three different fine-tuning methods are introduced for the pruned\ \ networks to reduce performance loss. SPADE is evaluated on compressed sensing\ \ MRI and image super-resolution, and is shown to speed up inference with minimal\ \ performance degradation.\n\nweaknesses and questions: Weaknesses:\n- The contributions\ \ of the paper are mostly comprised of a combination of existing techniques such\ \ as the pruning algorithm and the fine-tuning techniques.\n- The method is not\ \ compared with other methods for improving inference speed, such as [1] or [2]\ \ mentioned in the paper. The lack of this comparison makes it difficult to quantify\ \ the significance of the results. As an example, there is a 0.77 dB PSNR drop\ \ with a 51% speed up at test-time (Table 1) for compressed sensing MRI which\ \ seems to be a large performance reduction, and it is unclear how this compares\ \ to existing methods.\n\n[1] J. Liu, Y. Sun, W. Gan, X. Xu, B. Wohlberg, and\ \ U. S. Kamilov. SGD-Net: Efficient Model-Based\nDeep Learning With Theoretical\ \ Guarantees. IEEE Trans. Computational Imag., 7:598–610,\n2021.\n\n[2] J. Tang\ \ and M. Davies. A fast stochastic plug-and-play ADMM for imaging inverse problems.\ \ arXiv\npreprint arXiv:2006.11630, 2020.\n- How much does the training time increase\ \ for SPADE, compared with the baseline unpruned model-based network?\n- Is it\ \ possible to combine fine-tuning losses, rather than view them as independent\ \ techniques, and could that help preserve performance?\n- How does the memory\ \ complexity change at test-time? Memory complexity is also a quite important\ \ consideration for which discussion has not been included.\n\nSuggestions:\n\n\ - The introduction, and the \"DL and MBDL.\" subsection in the background are\ \ repetitive. For instance, the equation for PnP/RED does not seem to contribute\ \ to the story of the paper. The background can be shortened to include more experiments\ \ in the main paper, such as the visual results (Figure 6-8) in the supplemental,\ \ which are crucial for compressed sensing MRI. \n- Typographical errors should\ \ be fixed via proofreading." - "title: FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion\n\nsummary:\ \ This paper proposes an MOE-based model to handle multimodal data fusion. It\ \ addresses two challenges: missing modalities and irregularly sampled data trajectories.\ \ A Laplace gating function is applied to the MoE Backbone. An entropy regularization\ \ loss is proposed to ensure balanced and stable expert utilization. The author\ \ validates the method in diverse datasets.\n\nweaknesses and questions: Addressing\ \ the following weaknesses may improve the paper: \n\n1. The author should clearly\ \ distinguish the proposed method and others’ modules. From the paper, the Laplace\ \ gating function is proposed as a new one. I am not sure the author made some\ \ contributions to the encoder design, router design, and loss design. The author\ \ should make more illustrations about the contributions, not just combine other\ \ people’s work together. \n\n2. The experimental results are not extensive. The\ \ author should clearly demonstrate the data modality of the chosen benchmarks.\ \ It seems that other modalities, such as text, are not included. The author should\ \ explain this. \n\n3. Please explain more concisely how the gating functions\ \ can stabilize the imbalance and sparse multi-modal data. The paper should point\ \ this out more concisely and better with experimental results. Too many mathematical\ \ proofs and theorems in Sec.3 seem not helpful in illustrating the advantages\ \ of the proposed method.\nPlease see the weakness part to answer the questions." - "title: Compute Where It Counts: Adaptive Compute Allocation for Large Language\ \ Models via Learned Granular Sparsity\n\nsummary: CWIC (Compute Where it Counts)\ \ introduces a novel method for training sparse LLMs by making sparsity thresholds\ \ learnable parameters. The key contributions are: (1) learned contextual thresholds\ \ that are optimized using STEs, allowing models to dynamically allocate different\ \ amounts of compute to different tokens and weight matrices, and (2) granular\ \ sparsity that partitions matrix columns into smaller \"stripes\" for more expressive\ \ sparsity patterns.\n\nweaknesses and questions: Major Concerns\n\n1) I am not\ \ sure about the claim - *STE improves performance by removing the variance imparted\ \ on the grads when the values of G fluctuate...*: Using STE seems overly aggressive\ \ for the use-case? I think there needs to be more justification around this choice.\ \ Supporting experiments to compare that this is as a winning choice might also\ \ be helpful. Right now, this seems to more like a empirical tuning based selection?\n\ - If this choice is derived from JumpReLU - their gradient estimator worked for\ \ SAE reconstruction doesn't automatically validate these additional modifications\ \ for multi-layer distillation. Current explanation is not sufficient for a convincing\ \ argument.\n\nIn general, I think section 3.3 is poorly written. Authors should\ \ re-word their ideas more clearly here. \n\n2) No analysis of why related kernel\ \ choice is better than alternatives?\n\n3) The psuedo-derivative bandwidth (lines\ \ 220-224): The interaction between adaptive bandwidth $(\\epsilon_i = \\alpha\\\ epsilon · std(x_i))$ and input whitening (Section 3.4) is unclear - is std computed\ \ on whitened or raw values? How does this affect gradient scales?\n- No comparison\ \ of adaptive vs. fixed bandwidth for pseudo-derivatives.\n\n4) Experimental concern:\ \ The distillation data includes benchmark training sets (MMLU, ARC, WinoGrande)\ \ repeated 5×, which may inflate evaluation scores (refer: appendix C I believe).\ \ I think reporting scores on benchmarks not in training data would be a stronger\ \ claim.\n\n5) Loss function: Authors need to explain how to interpret this. Right\ \ now, it looks asymmetrical: the loss only penalizes using too many parameters,\ \ not too few? In other words, it will keep on minimizing, i.e nothing stops the\ \ model from becoming 10× sparse if that happens to minimize distillation loss?\ \ It seems one would need careful tuning/warm starting to counter this or is there\ \ a implicit effect from distillation loss (or am I understanding this wrong)?\n\ \nI strongly think the presentation wrt to the key ideas (section 3.2-3.6) need\ \ to be improved to make the author's work more accessible and make a convincing\ \ argument towards a lot of different empirical choices. Additionally, I found\ \ the concatenation operator to be a bit confusing (line 180-181), maybe there\ \ is a better way to mathematically denote it?\nNA" - "title: Unified Insights: Harnessing Multi-modal Data for Phenotype Imputation\ \ via View Decoupling\n\nsummary: This paper focuses on the task of phenotype\ \ imputation and proposes utilizing multi-modal data to gain insights that facilitate\ \ the evaluation of patients' overall health status. Specifically, the authors\ \ design a framework based on view decoupling, which involves segregating the\ \ modeling of biological data and phenotype data to avoid the impact of data heterogeneity\ \ and view conflict. To alleviate the influence of noise and irrelevant information\ \ in the biological data, a novel contrastive knowledge distillation method is\ \ proposed. Furthermore, the authors conduct extensive experiments to demonstrate\ \ the superiority of the proposed model.\n\nweaknesses and questions: 1.\tThe\ \ model includes multiple components. It would be beneficial to discuss the time\ \ complexity of the proposed method. Specifically, an analysis of the computational\ \ efficiency for each component, as well as the overall model, would provide valuable\ \ insights.\n2.\tThe patients in the experiments are selected from those with\ \ Alzheimer's disease and related dementias. It would be helpful to explain the\ \ rationale behind selecting this particular patient set. Additionally, it is\ \ important to discuss whether the model is applicable to other cohorts.\n3.\t\ Why can't recent models, such as M3Care, Graph, and MUSE, directly address the\ \ need for integrating biological data and EHR data?\n4.\tThe proposed method\ \ involves multiple loss functions. Adding these losses to Figure 1 would aid\ \ understanding. Including pseudocode for the algorithm would also be helpful.\n\ Please refer to weaknesses." - "title: Distributed Parallel Gradient Stacking(DPGS): Solving Whole Slide Image\ \ Stacking Challenge in Multi-Instance Learning\n\nsummary: This paper introduces\ \ Distributed Parallel Gradient Stacking (DPGS), a framework designed to address\ \ the challenge of non-stackable data in Multiple Instance Learning (MIL) for\ \ Whole Slide Image (WSI) analysis. The authors propose two key components: (1)\ \ DPGS, which enables parallel processing of variable-length MIL bags by distributing\ \ them across multiple GPUs and aggregating their gradients, and (2) Deep Model-Gradient\ \ Compression (DMGC), which reduces communication overhead during distributed\ \ training through joint compression of gradients and model parameters. Experiments\ \ on Camelyon16 and TCGA-Lung datasets demonstrate significant improvements in\ \ both training speed (up to 31× faster) and classification accuracy (up to 9.3%\ \ increase) compared to baseline methods.\n\nweaknesses and questions: Strengths:\n\ 1.\tThe mathematical derivations showing equivalence to mini-batch training are\ \ sound.\n2.\tThe gradient compression approach (DMGC) offers an interesting extension\ \ to existing gradient compression techniques.\n3.\tThe ablation studies provide\ \ useful insights into the factors affecting performance within their framework.\n\ Weaknesses:\n1.\tHigh resource requirements: Although the speed-up performance\ \ is impressive, the method requires multiple GPUs and high-bandwidth connections,\ \ significantly limiting its practical applicability in many research and clinical\ \ settings.\n2.\tMissing comparisons to simpler alternatives: The paper doesn't\ \ evaluate simple alternatives such as uniform sampling that could potentially\ \ achieve similar results with much less complexity.\n3.\tOutdated baselines:\ \ The paper relies on comparisons with older MIL methods rather than current state-of-the-art\ \ approaches.\n4.\tUnclear feature extraction: The \"multi-scale\" features referenced\ \ throughout the paper are not adequately defined.\n5.\tLimited relevance given\ \ foundation models: The paper doesn't acknowledge or compare against foundation\ \ models that have demonstrated near-perfect performance on the same datasets.\n\ 1.\tCould you provide a clear definition of the \"multi-scale\" feature extraction\ \ process used in your experiments, including architectures and implementation\ \ details?\n2.\tHave you compared your approach with simpler methods like uniform\ \ sampling from each WSI to create fixed-length bags that could be trained with\ \ standard batch processing?\n3.\tCould you explain your experimental setting\ \ and the reason why you did not consider current foundation model-based approaches\ \ (UNI [1], CONCH [2], PLIP [3], etc.) that have demonstrated state-of-the-art\ \ performance on the same datasets?\n4.\tWhat is the minimum hardware configuration\ \ required to achieve meaningful benefits from your approach compared to single-GPU\ \ training?\n5.\tWhy did you choose to compare against older MIL methods rather\ \ than more recent approaches that might represent stronger baselines?\nI will\ \ consider raising the overall recommendation score if these questions are solved\ \ in the rebuttal phase.\n[1] Chen, Richard J., et al. \"Towards a general-purpose\ \ foundation model for computational pathology.\" Nature Medicine 30.3 (2024):\ \ 850-862.\n[2] Lu, Ming Y., et al. \"A visual-language foundation model for computational\ \ pathology.\" Nature Medicine 30.3 (2024): 863-874.\n[3] Huang, Zhi, et al. \"\ A visual–language foundation model for pathology image analysis using medical\ \ twitter.\" Nature medicine 29.9 (2023): 2307-2316." - source_sentence: quantization prediction flips sentences: - 'title: Block Rotation is All You Need for MXFP4 Quantization summary: This paper presents an investigation into the applicability of existing post-training quantization (PTQ) methods for LLMs under the emerging MXFP4 format. The core finding that global rotation-based methods are fundamentally incompatible with MXFP4''s block-wise scaling, thus they posit that a simple block-wise rotation (BRQ) is able to mitigate this issue. They provide extensive experiments and analysis. While the topic is timely, the paper suffers from a fundamentally incremental contribution and a lack of technical novelty that is enough to question its value to the quantization community. The core insight is deemed obvious for the target audience, and the solution does not constitute a significant algorithmic advance. weaknesses and questions: 1. The central problem and its solution are a straightforward, expected outcome for anyone with deep expertise in quantization. Applying a block-level transformation to align with a block-level quantization scheme is a natural and almost trivial engineering adjustment, not a novel research contribution. The MXFP4 format, by design, uses local block scaling (PoT) to contain outliers. Applying a global operation that deliberately spreads out outlier energy directly counteracts the format''s core design principle. Therefore, observing a performance collapse is not a discovery; it is a confirmation of a predictable hardware/algorithm mismatch. 2. The proposed BRQ method is a direct and obvious application of existing concepts. It simply restricts the well-known rotation transform to the block granularity defined by the hardware. This does not represent a new algorithm or a conceptual breakthrough. 3. They selected ''datasets and benchmarks'' as the primary area, but there is no new datasets or benchmarks provided. NA' - "title: Robust Training of Neural Networks at Arbitrary Precision and Sparsity\n\ \nsummary: Quantization aware training (QAT) is a widely used method to prepare\ \ models for quantization. To overcome the rounding operation during backward\ \ pass, QAT uses straight through estimator (STE) which bypasses the gradients\ \ in the backward pass outside of the rounding operation. This work, characterizes\ \ this feature as the backward pass being quantization oblivious. To overcome\ \ this, the authors present a workaround that exposes the backward pass to the\ \ quantization error using a combination of ridge regression and affine quantization.\ \ Experiments on a wide array of models, and bit regimes shows superior performance\ \ of the proposed method when compared to plain STE-based QAT.\n\nweaknesses and\ \ questions: * **QAT and error minimization:** The fundamental hypothesis of this\ \ work is that during the backward pass, QAT is oblivious to the quantization\ \ error. While this is possible, it has been shown by now that STE creates a different\ \ type of dynamic that results in weight oscillations. And these oscillations\ \ have factors that are trying to compensate the errors due to QAT [1,2]. Given\ \ this, what do the authors make of these explanations? And how does it alter\ \ their hypothesis, or not? If not, what is their argument?\n\n* **STE beyond\ \ 2 bits:** Even if one were to attribute STE for poor performance in extremely\ \ low-bit width, how is this not manifested to the same degree in higher bit regimes.\ \ What is the explanation? Is STE more problematic only in extremely low-bit regimes?\ \ \n\n* **Ternary quantization with STE:** There are several works that use STE\ \ for ternary quantization [3,4]. The critique that STE cannot be used for extremely\ \ low-bit regimes does not hold up. \n\n* **Presentation clarity:** The ideas,\ \ experiments, and results are quite compelling in this work. However, the presentation\ \ is unclear in many places. There are vague statements, unsubstantiated by evidence\ \ (discussions around biological neurons, intelligence), and presentation of results\ \ makes it difficult to parse them. Many of the interesting results are in the\ \ Appendix (Fig. 3, Table 1) whereas the main results in Fig. 1 and Fig. 2 are\ \ illegible, with no clear captions, legends, axes labels. This is unfortunate\ \ as it dilutes the impact of otherwise nice contribution. I would suggest improving\ \ these aspects. \n\n* L-67: Very vague statement with exaggerated claims. It\ \ is by now common knowledge that large, quantized models outperform smaller ones.\ \ And also the claim of biological intelligence is extremely misplaced. \n\n###\ \ Other comments\n\n* Reference to Figure 1-a in L-37 is not useful as none of\ \ the concepts are fully introduced; consider dropping this reference or elaborating\ \ the caption so that it can independently explain the concepts in the figure.\n\ \n* L-38: What are the heuristic-based modifications authors are pointing to?\ \ No references to back this up. \n\n* L-54: Strange sentence; perhaps missing\ \ a preposition somewhere. Did the authors mean \"full potential of the theoretically...\"\ \n\n### References\n\n1. Wenshøj, Jonathan, Bob Pepin, and Raghavendra Selvan.\ \ \"Oscillations Make Neural Networks Robust to Quantization.\" arXiv preprint\ \ arXiv:2502.00490 (2025).\n2. Xie, Weiying, et al. \"Allowing Oscillation Quantization:\ \ Overcoming Solution Space Limitation in Low Bit-Width Quantization.\" Proceedings\ \ of the IEEE/CVF International Conference on Computer Vision. 2025.\n3. Choi,\ \ Jungwook, et al. \"PACT: Parameterized Clipping Activation for Quantized Neural\ \ Networks.\" (2018).\n4. Wang, Jinheng, et al. \"1-bit ai infra: Part 1.1, fast\ \ and lossless bitnet b1. 58 inference on cpus.\" arXiv preprint arXiv:2410.16144\ \ (2024).\nSee weaknesses above." - "title: Nearly Lossless Adaptive Bit Switching\n\nsummary: The paper addresses\ \ challenges in model quantization for deep neural networks (DNNs), focusing on\ \ optimizing quantization-aware training (QAT) across multiple bit-widths with\ \ weight-sharing. To this end, this paper introduces a novel quantization method\ \ that exploits the highest integer precision to achieve nearly lossless bit-switching,\ \ reducing storage without relying on full precision. Key contributions include:\ \ (1) Adaptive Learning Rate Scaling: A technique that dynamically adjusts learning\ \ rates for different precisions to address competitive interference and inconsistent\ \ gradient issues during one-shot joint training. (2) Double Rounding: An extension\ \ for one-step rounding quantizer in fixed-precision quantization to improve accuracy.\ \ Experimental results on the ImageNet-1K dataset show that the proposed methods\ \ surpass state-of-the-art approaches in both multi-precision and mixed-precision\ \ scenarios, achieving higher efficiency and accuracy.\n\nweaknesses and questions:\ \ - Some analysis is missing. For example, I'm wondering whether the second rounding\ \ leads to more quantization errors, as the first rounding is used to produce\ \ INT8 weights and second rounding is then performed to quantize lower bit-width,\ \ the twice quantization is possible to cause more clipping errors and rounding\ \ errors, some analysis could enhance the strength of proposed methods. \n- Some\ \ designs should be further clarified, e.g., why ALRS is applied only for the\ \ scaling factors? Intuitively, weights of small bit-width is induced large gradient\ \ variance by STE, and thus the weights of small bit-width should also benefit\ \ from using smaller LR. \n- Fig. 1 is a bit confusing, some colored arrows are\ \ not well explained. \n- This works essentially lies in the research of mixed-precision\ \ quantization, so I think it is better to compare more MPQ (e.g., HAQ, DNAS,\ \ LIMPQ, etc) research in the Sec.4. Moreover, some recent papers on multi bit-width\ \ quantization are missed on the , e.g., [1] (PTQ-based) and [2][3] (QAT-based),\ \ which could be included into the Related Work. \n\n[1] Xu, Ke, et al. \"PTMQ:\ \ Post-training Multi-Bit Quantization of Neural Networks.\" Proceedings of the\ \ AAAI Conference on Artificial Intelligence. Vol. 38. No. 14. 2024.\n\n[2] Tang,\ \ Chen, et al. \"Retraining-free model quantization via one-shot weight-coupling\ \ learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern\ \ Recognition. 2024. \n\n[3] Zhong, Yunshan, et al. \"MultiQuant: A Novel Multi-Branch\ \ Topology Method for Arbitrary Bit-width Network Quantization.\" arXiv preprint\ \ arXiv:2305.08117 (2023).\nPlease refer to the weaknesses." - 'title: FPTQuant: Function-Preserving Transforms for LLM Quantization summary: This paper introduces FPTQuant, a novel PTQ framework designed to mitigate the severe activation outliers in LLMs. The authors cleverly design four lightweight FPTs—including a mathematically elegant pre-RoPE transform and MLP scaling techniques—that smooth activation distributions without altering the exact mathematical outputs of the unquantized model. By co-optimizing these transforms locally and end-to-end, FPTQuant achieves highly accurate static INT4 quantization with virtually zero inference overhead, completely bypassing the need for slow, bespoke CUDA kernels. weaknesses and questions: Strength: 1. The block-diagonal pre-RoPE transform is good. Instead of fighting the architecture, exploiting the equivariances of modern Transformers to commute with the RoPE matrix is exactly the kind of principled, math-first solution the quantization community needs right now. 2. This method folds transforms directly into the weights. Avoiding writing CUDA kernels for non-standard hardware is crucial for real-world deployment and cross-hardware compatibility. Weakness: 1. Claims of "virtually no overhead" are great, but theoretical FLOP reduction isn''t enough. Hardware metrics are needed—like e2e latency, TTFT, and decode throughput (Tokens/sec) across varying batch sizes on standard GPUs (A100/H100)—to prove that memory bandwidth isn''t secretly being bottlenecked. 2. Because FPTQuant requires both local and end-to-end training of the transforms, what is the actual wall-clock time and peak GPU memory required for the calibration phase? Especially compared to other training-based pts methods like spinquant. 3. Real-world INT4 usually relies on group-wise quantization (e.g., group size 64/128) to maintain accuracy. Since the FPTs use channel-wise scaling to tame activation outliers, they inevitably alter the weight variance within those specific groups. It''s a noticeable gap that the paper doesn''t analyze—either theoretically or empirically—whether these transforms inadvertently blow up intra-group weight variance. If so, this would severely degrade group-wise quantization fidelity. See weakness' - 'title: Large Language Models do Not Make Complete Use of Math Reasoning Data summary: The paper studies per-item generalization when fine-tuning LLMs on math-reasoning tasks. As training data is incrementally increased, many test items that were previously answered correctly flip to incorrect, so net accuracy improves only marginally because “newly correct” and “newly incorrect” items roughly cancel. The finding is demonstrated for SFT (e.g., Llama-3-8B, Gemma-3-4B on GSM8K/MAWPS) and RL (Qwen2.5-0.5B on GSM8K/MATH8K). The authors quantify a “Union vs Final” gap (items solved by any intermediate model vs the final model), show cross-seed “predictive multiplicity” (same data, different seeds → different test items solved), and offer a high-level explanation via a “strategy set” view of reasoning traces. Ablations suggest sample order and LoRA dropout contribute to divergence across seeds. No new training method is proposed; the work is diagnostic/observational. weaknesses and questions: – Limited causal explanation: the paper establishes that flips occur, but the proposed “strategy set/Rashomon” lens is descriptive and not predictive; it does not isolate necessary/sufficient causes of flipping or quantify their contributions (e.g., data conflicts vs optimization noise vs under/over-fitting vs decoding effects). – Scope is narrow (math-reasoning, modest model sizes); it is unclear how broadly the phenomenon holds (code, multilingual, instruction-following, safety, etc.), or how it scales with much larger base models and longer training. – No actionable solution: beyond noting order/dropout effects, the work stops short of proposing methods to reduce flips or close the Union-vs-Final gap (e.g., curriculum, data reweighting, conflict detection, checkpoint ensembling, order-invariant updates, strategy-diversity regularizers). – Experimental controls are thin in places: small number of seeds; limited statistical testing; compute/training-length/early-stopping effects not deeply probed; decoding settings (e.g., temperature/self-consistency) only partially explored. – The “union” signal naturally suggests simple mitigations (checkpoint ensembling across data-subset steps, EMA over training, mixture-of-checkpoints) that are not tried; without testing such baselines, the practical impact remains unclear. – Theoretical component does not yield falsifiable predictions (e.g., when flips should increase/decrease given measurable dataset/model properties). • What fraction of flips can be attributed to measurable data conflicts (near-duplicates with differing rationales/solutions, annotation noise) vs optimization stochasticity? Can you quantify this via conflict detection or per-sample gradient similarity analyses? • Can you predict which items will flip when adding data? For instance, are low-margin items (by log-prob gap), longer reasoning chains, or particular operation types more flip-prone? • Does the phenomenon persist with substantially larger base models and longer training to convergence? How does it scale with training steps/epochs and gradient noise scale? • What is the effect of decoding schemes (temperature, self-consistency, verifier-guided selection, tool-use) on flips beyond majority voting? • Do simple mitigations narrow the Union-vs-Final gap: (a) checkpoint ensembling across subset steps, (b) EMA of weights, (c) curriculum or order-invariant batching, (d) removing LoRA dropout and fixing order for all conditions, (e) data deduplication/cluster-balanced sampling? • Does full-parameter fine-tuning (no adapters) or different PEFT choices alter the flip rate? • Can the strategy-set view be made predictive (e.g., estimating effective strategy entropy per item) and tested against flip rates?' - "title: Accuracy is Not All You Need\n\nsummary: The authors analyze changes in\ \ model predictions as an additional performance metric to evaluate quantization\ \ schemes applied to large language models (LLaMa2 and Yi). The authors show that\ \ during model quantization that the predictions for a substantial number of examples\ \ change, beyond the limited number of examples which labeled incorrect post quantization.\ \ Additionally, the authors show that the number of examples that change in prediction\ \ is correlated with the KL-Divergence between the base and quantized model.\n\ \nweaknesses and questions: The overall results are unsurprising and do not provide\ \ additional characterizations of the effects of quantization on model performance\ \ or example characteristics.\n- It is expected that model predictions with lower\ \ confidence are more likely to change upon compression. However, the current\ \ analysis does not provide insight into how these examples change during compression.\ \ For example it is not clear if the reason that low confidence examples change\ \ in prediction is because they are initially close to the decision boundary or\ \ if they observe greater change in model confidence during compression.\n- Change\ \ in model prediction has been studied both in the context of continual learning\ \ and interpretability analysis of model training dynamics [1,2]\n\nRelated Work\n\ 1. Toneva, Mariya, et al. \"An Empirical Study of Example Forgetting during Deep\ \ Neural Network Learning.\" International Conference on Learning Representations.\ \ 2018.\n2. Swayamdipta, Swabha, et al. \"Dataset Cartography: Mapping and Diagnosing\ \ Datasets with Training Dynamics.\" Proceedings of the 2020 Conference on Empirical\ \ Methods in Natural Language Processing (EMNLP). 2020.\n1. Are the flipped examples\ \ consistent across different compression, quantization schemes, or models? \n\ 2. For a single method scheme with varying degrees of compression (GPTQ W8A16\ \ -> GPTQ W4A16; or pruning at different intensities), do the flips increase monotonically,\ \ consistenlty with increasing degrees of quantization? For example, are the examples\ \ that are forgotten at 8-bit quantization relearned at 4-bit quantization?\n\ 3. The top margin (difference in probability) metric is used to characterize the\ \ model performance pre-quantization; A. What does the metric indicate post quantization?\ \ B. Does quantization result in a more uniform probability distribution? C. Are\ \ the shifts in predicted likelihood and confidence larger or smaller for examples\ \ that were initially higher or lower in confidence?" - 'title: Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models summary: This paper provides a comprehensive empirical study on the quantization of reasoning models. They evaluate SoTA quantization methods across weight-only, weight-activation, and KV cache quantization. Their conclusion is: Lossless settings: W8A8 or W4A16 are consistently < 1 % accuracy drop; W4 weight‑only or KV‑cache is “near‑lossless” on most models. Best algorithms: AWQ for weight‑only, QuaRot for KV‑cache (except small Qwen), FlatQuant for weight‑activation. Failure modes: 3‑bit quantization, small models, harder tasks and RL‑trained models (QwQ) are much more fragile. No extra thinking: mild quantization does not lengthen CoT while aggressive low‑bit settings do. Scaling trade‑offs: 4‑bit large models beat smaller BF16 models in both accuracy‑per‑GB and accuracy‑per‑second. weaknesses and questions: 1. No novel ideas are proposed. It is more about empirical study paper. The main contribution of this paper is doing experiments for the current quantization algorithms, and conclusion might be change as new approaches evolve. This paper would be benefit from this by proposing a leaderboard that support evaluations of any quantization algorithms. 2. It does not really explain why certain quantization algorithms fail or do well. For example, it states "AWQ is preferred over GPTQ", "QuaRot is generally preferred over KVQuant*". This paper could be benefit if author can connect those empirical conclusions (accuracy drop) with some theoretically justifications and provide some insights in designing new algorithms.' - 'title: Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction summary: MXFP4 and NVFP4 exhibit a significant gap in fidelity. The authors trace MXFP4''s lower accuracy to its rigid power-of-two scaling factors and coarser block granularity, which struggle to preserve outliers. To mitigate this without requiring hardware modifications, the authors introduce two software-only techniques, Overflow-Aware Scaling (OAS) and Macro Block Scaling (MBS), that improve MXFP4 quantization fidelity without requiring hardware changes. These methods reduce the gap between MXFP4 and NVFP4 to roughly one-tenth of its original size. weaknesses and questions: Strengths: 1. The problem addressed in this paper is important and impactful, and has significant value for the community. 2. The experimental results are substantial: the average accuracy gap is reduced from 10% to 1%. Weaknesses: I am not very familiar with this field. The methods appear to be sound. Other weaknesses and limitations can be referenced in the questions below. 1. The evaluation appears to omit calibration-based post-training quantization baselines. Modern deployments typically rely on algorithms such as GPTQ, AWQ, or SmoothQuant to handle outliers before format conversion. If weights and activations have already been smoothed, how much additional benefit would the proposed methods provide? 2. Could the authors provide token generation latency benchmark results? 3. Could the authors include a histogram showing the frequency with which the OAS threshold is triggered?' - source_sentence: semantic labels data efficient learning sentences: - 'title: Language Semantic Graph Guided Data-Efficient Learning summary: The paper introduces the Language Semantic Graph (LSG), a novel approach to data-efficient learning that leverages semantic information from labels. The LSG is used to train an auxiliary graph neural network, which then guides the primary model''s training, enhancing the utilization of label knowledge. This method is applicable across various modalities, including image, video, and audio, and has shown significant performance enhancement in both Transfer Learning and Semi-Supervised Learning scenarios. Experiments were conducted on seven standard datasets covering images, videos, and audios, using several deep neural networks with different architectures and pretraining datasets. The results show that LSG significantly outperforms other methods, especially when labeled data is scarce. It also demonstrates promising potential in semi-supervised settings, achieving the best performance across all labeling rates and datasets. When applied to self-supervised pretrained models, LSG shows consistent gains. It also improves model performance on both in-distribution and out-of-distribution samples, indicating that label semantic relations help the model learn more robust features. In video and audio experiments, LSG consistently improves the fine-tuning accuracy across all tasks with limited labeled samples. It outperforms other methods, boosting accuracy significantly. For audio experiments, LSG achieves an average of 5.56% accuracy enhancement from the baseline, demonstrating its wide applicability across various modalities. "LSG consists of two parts: an auxiliary graph neural network that extracts knowledge from the semantic graph and two novel optimization objectives that transfer the knowledge to primary models." The authors demonstrate that LSG is applicable on image, video and audio models and brings significant performance gains to the model under Transfer Learning and Semi-Supervised Learning scenarios. weaknesses and questions: The effectiveness of the LSG method relies heavily on the quality and semantic richness of the labels. In scenarios where labels are sparse, ambiguous, or not well-defined, the performance of the LSG method could be compromised. *The LSG method relies heavily on the quality and semantic richness of the labels. How does the quality of the labels impact the performance of the LSG method? Could the LSG method be adapted to work effectively with less informative or ambiguous labels, and if so, how? * The paper primarily focuses on classification tasks. Could the LSG method be adapted or extended to other tasks, and if so, what modifications would be necessary?' - "title: Concept Bottleneck Models under Label Noise\n\nsummary: The paper is concerned\ \ with inherently interpretable models called Concept Bottleneck Models (CBMs).\ \ These models require extensive concept labeling; however, these labels are usually\ \ assumed to be perfect. The paper explores how noise in these concept labels\ \ affects the final target prediction. The authors perform extensive experiments\ \ across all three variants of CBMs to show the detrimental effects of concept\ \ labels. The authors then proposed SAM training to improve concept and target\ \ accuracy.\n\nweaknesses and questions: - While the experiments are comprehensive\ \ for Section 3 and 4. Some of the results are pushed to the appendix (which is\ \ fine), however it would have been nice to summarise them in brief in the text.\n\ \n- I enjoyed reading up to Section 4. Thank you. However, I would have appreciated\ \ some theoretical intuition on why SAM works better (unless I missed it).\n\n\ - The paper performs experiments with CUB and AwA2 datasets, popular benchmark\ \ datasets for CBMs. These datasets however have a strong correlation between\ \ concept labels and the target label. I can imagine label noise to be very detrimental\ \ (as observed from Figure 2). The potential observed effect due to label noise\ \ might be weaker in the case of diverse concepts for each target.\n\n- The paper\ \ claims to be first the paper to looking at label noise in CBMs, while I would\ \ not refute this, I would like to point out the authors to some very relevant\ \ papers - [1] (noise added to concept labels, similar to some of the exps in\ \ Sec3) [2]-(concept robustness and adv attacks)\n\n[1] - Sheth, Ivaxi, and Samira\ \ Ebrahimi Kahou. \"Auxiliary losses for learning generalizable concept-based\ \ models.\" Advances in Neural Information Processing Systems 36 (2024).\n\n[2]\ \ - Sinha, Sanchit, et al. \"Understanding and enhancing robustness of concept-based\ \ models.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol.\ \ 37. No. 12. 2023.\n- In Sec 6.1 authors show that other CBM variants are also\ \ susceptible to label noise. This alings with CBMs, however the obvious question\ \ for me is, does SAM training improve the robustness? Why or Why not?\n\n- Concept\ \ labeling CBMs is very difficult, there is an increasing interest in using LLMs\ \ for concept annotation. Can the authors kindly comment (maybe in Limitations\ \ section of the paper), on whether label noise will impact such concept labels.\ \ \n\n- Interventions are useful aspect of CBMs. What is the impact of interventions\ \ to reduce label noise? I assume interventions may be less effective. Does SAM\ \ improve it? \n\n\nMinor:\n\n- Line 101, now caps for \"We\".\n\n- Figure 3,\ \ which model is used not specified? Joint/Ind/Seq?" - 'title: DREAM: Dual-Standard Semantic Homogeneity with Dynamic Optimization for Graph Learning with Label Noise summary: This paper addresses the important and timely problem of robust graph learning under label noise, proposing a dynamic reweighting scheme, DREAM, that integrates relational information. During training, it dynamically reweights labeled nodes using semantic homogeneity scores. This method is well-motivated and potentially applicable to many graph learning settings. Its dual-criteria anchor selection mechanism combines proximity-aware and topology-aware anchors. Experimental results show that, on standard benchmark datasets and noise models, this method achieves significant and stable improvements compared to several strong baselines. weaknesses and questions: Strengths: * Semantic homogeneity scores based solely on graph structure relationships, rather than solely on loss values or simple auxiliary regularization terms, provide a direct, relation-informed reliability signal. * This method addresses a well-known problem in node classification under label noise: instead of relying solely on small-loss heuristics or global consistency terms, it explicitly utilizes graph structure relationships to determine which labels are trustworthy. This method can be integrated with standard GNN backbones, showing potential value for practical applications in handling noisy graph labels. * Experiments were conducted on six benchmark datasets and three noise types. Compared to a series of LLN and GLN baseline methods, the proposed method achieves significant and stable improvements, and ablation experiments validate the effectiveness of each module. Weaknesses: * The theoretical guarantee of this method requires that the semantic homogeneity score $H(n_i;A(n_i);\tau)$ must be a $(\beta,\epsilon)$ approximation of the ideal importance weight $\alpha(x_i,\hat{y}_i,y_i)$. However, this paper neither provides the conditions for this approximation to hold nor offers strong quantitative evidence that $H$ can reliably reflect the cleanliness of the labels. * Proximity-aware anchors relies on noisy labels and is prone to confirmation bias. The nearest neighbor candidate set $C_P(n_t)=\{n_i\mid i\in S \wedge y_i=y_t\}$ is constructed based on observed labels (which may have been contaminated). In cases of high noise rates, small sample classes, or weak representations in the early stages of training, anchor point selection may be biased towards nodes that have been mislabeled as "same kind," reinforcing erroneous supervision signals before the embedding becomes discriminative. * Equation (2) implicitly assumes that the candidate pool is large enough to select exactly $k_P$ nearest neighbor anchors, but the paper does not explain how to handle the situation where $|C_P(n_t)| 2 mapping is\ \ excessive. The authors should claim what is the advantage of using RL approaches\ \ compared to the simple adaptive thresholding like [1, 2].\n\n- While MLLM and\ \ SAM2 are used offline, the overhead of generating priors for all unlabeled data,\ \ plus the RL agent's training cost, is never reported. \n\n[1] FlexMatch: Boosting\ \ Semi-Supervised Learning with Curriculum Pseudo Labeling\n\n[2] Adaptive Self-training\ \ Framework for Fine-grained Scene Graph Generation\n- Can you add natural heuristic\ \ baselines—using pseudo-labels only where both the MLLM prior and model prediction\ \ are confident (intersection-based filtering). \n\n- Given that the state space\ \ is 6D and action space is 2D, have you compared RPLE against simpler adaptive\ \ thresholding baselines? \n\n- The Baseline (w/ MLLM) uses a fixed mixing coefficient\ \ of 0.5. What happens when this coefficient is optimized via grid search on the\ \ validation set?" - "title: Learning with Selectively Labeled Data from Multiple Decision-makers\n\ \nsummary: This paper focuses on the problem setting of classification with selective\ \ labeled data, that is, the labeled data at hand can be biased because of decision-making\ \ in the past. This paper defines the problem mathematically and solves this problem\ \ from the perspective of the instrumental variable (IV) framework. There are\ \ two assumption settings: (1) No unmeasured common effect modifiers (NUCEM),\ \ which is a strong assumption that leads to a clean solution, and (2) Partial\ \ identification, where a reasonable solution can be obtained. Theoretical analyses\ \ of two assumption settings are provided. Furthermore, a practical algorithm\ \ for both cases is also provided based on weighted empirical risk minimization\ \ with calibration guarantee. Synthetic experiments show that the proposed method\ \ outperforms baselines.\n\n## update after rebuttal\nAfter the rebuttal, I still\ \ think the idea of this paper is novel. It studies the problem setting extensively\ \ theoretically and also provides experimental results., Thus, I keep my score\ \ (4: accept). The authors clarified in the rebuttal that their work has novelty\ \ and also admits some current drawbacks of their methods (e.g., computation time).\n\ \nweaknesses and questions: Strengths\n1. Strong theoretical results that improve\ \ an understanding of a complicated yet relevant problem setting. It is praiseworthy\ \ that this paper not only focuses on a restrictive NUCEM assumption but also\ \ considers the partial information assumption.\n2. Practical algorithms with\ \ theoretical guarantee are provided, which can be relatively easy to implement.\ \ \n3. Experimental results (although synthetic) show that the proposed method\ \ is effective compared with reasonable baselines.\n\nWeaknesses\n1. Proposed\ \ method's weakness is not much discussed in my understanding. One might be that\ \ it could be computationally expensive (I'm not sure). Moreover, the estimation\ \ of weight could be incorrect, and we don't see much effect in the experiments,\ \ whether this can make the proposed method not work well. I find the comment\ \ in the paper, why NUCEM lost to partial under NUCEM assumption quite interesting\ \ that NUCEM requires a ratio estimation. I think such discussions could be useful.\ \ Or ablation study of the effect of imprecise weight estimation could also be\ \ useful. (but I'm also aware that the paper is already dense unfortunately).\n\ 1. Could you please comment on the comparison of the computational cost of the\ \ proposed unified cost-sensitive learning (point), (partial), and vanilla training?\n\ 2. Since many weights have to be estimated, how important is the accuracy of weight\ \ estimation? Is the solution highly sensitive to this?\n3. Is this the first\ \ work to use instrumental variable (IV) framework for selective labeled classification?" - "title: Hard Labels In! Rethinking the Role of Hard Labels in Mitigating Local\ \ Semantic Drift\n\nsummary: This paper proposes to utilize hard labels to both\ \ address the massive storage bottleneck and improve performance for dataset distillation\ \ methods. The paper identifies that reducing cropped views introduce a problem\ \ called local-view semantic drift, when crops are assigned soft labels that deviate\ \ from the ground-truth image label. To solve this, the paper proposes a 3-stage\ \ training scheme that consists of 2 soft-label stages and a key hard-label stage\ \ in the middle that theoretically improve the gradient alignment of the two label\ \ types. Massive theoretical analysis and a semantic drift framework are provided\ \ to support the method. Experiments demonstrate state-of-the-art performance\ \ and a 100x reduction of storage cost.\n\nweaknesses and questions: Strengths:\n\ \nThe theoretical framework of the soft-hard alignment is thorough in this paper,\ \ with in-depth analysis of the Local-View Semantic Drift problem studied in the\ \ paper. Meanwhile, the empirical validation shows strong effectiveness and efficiency.\n\ \nWeaknesses:\n- The main method, i.e., training stage B, seems not using hard\ \ labels for correcting signals, which is the primary motivation of the paper.\ \ The method basically uses label smoothing + Cutmix, both still creating soft\ \ labels.\n- The theoretical part before Sec.3.2 is quite disconnected to the\ \ main method.\n- The number of compared methods are limited. The authors are\ \ suggested to compare 1~2 more (e.g., GIFT).\n- The evaluated datasets are limited\ \ to only Tiny-ImageNet/ImageNet-1K, limiting it only in general domains.\n- Minor\ \ issues: \n - Notation of $s$ defined for the second time at page 4 stage A.\n\ \ - Table 6 caption: right/left rather than top/bottom.\n - The best-performing\ \ $\\alpha$ is annotated wrong in Table 13.\n1. Why is the one-hot labels not\ \ used for stage B directly?\n2. How is the LS + Cutmix method able to perform\ \ gradient correction?\n3. Is there intuitive explanation or empirical evidence\ \ on how the three stages work to bound the gap in Theorem 3.5?" - 'title: Semantic F1 Scores: Fair Evaluation Under Fuzzy Class Boundaries summary: This paper introduces Semantic F1 Scores, a family of evaluation metrics for multi-label classification that account for semantic similarity between labels. The core idea is to give partial credit when a predicted label is semantically related to the true label, instead of counting it as a complete miss as in standard F1. The metric uses a two-step matching (predict-to-true and true-to-predict) to compute “semantic precision” and “semantic recall,” which are combined into an F1 score. Semantic F1 is designed to be backward-compatible (it reduces to standard F1 when no partial credit is allowed). Additionally, to theoretical arguments, the authors conduct an empirical study on eight datasets (synthetic and real) and demonstrate that Semantic F1 provides potentially fairer and more informative evaluation in tasks with subjective or overlapping labels, correlating more closely with real-world outcomes than classical F1. weaknesses and questions: - I think the biggest weakness of the approach is dependency on the similarity matrix. If the matrix S is poorly specified or biased, scores could be misleading or unfair, which makes it tricky to use as an evaluation metric. The defining similarity may be non-trivial. Similarity derived from label co-occurrence or embeddings might not truly reflect conceptual closeness. Results with different S matrices are not comparable. - Baseline comparison on real tasks is limited, as on real datasets, evaluation focuses on hard F1 vs Semantic F1, with no direct comparison to alternative semantic metrics, which also feels limited in a synthetic study. - I''m generally a bit confused by experiments; it is not clear to me what is optimized and what is evaluated, and why should I care that Semantic F1 better correlates with the task than hard F1? And what about other soft/semantic metrics? - Many labels on Figures 2 and 3 are so small that they are not readable when printed. Please, see weaknesses session.' - "title: Beyond One-Hot Labels: Semantic Mixing for Model Calibration\n\nsummary:\ \ This paper propose Calibration-aware Semantic Mixing, a model calibration approach\ \ using diffusion-based data augmentation, like “semantic mixup”.Unlike traditional\ \ one-hot labeling, CSM generates mixed samples with soft labels with the CLIP.\ \ The authors introduce a reannotation technique using CLIP features and investigate\ \ the influence of loss functions prove L2 loss is good for enhances calibration.\n\ \nweaknesses and questions: Weaknesses: \n1. Limited discussion on computational\ \ efficiency\n2. Hyperparameter sensitivity analysis is not well explored.\n3.\ \ No Transformer model is included for comparison\n1. How does the choice of diffusion\ \ model affect the performance of CSM? How about we use other generative models.\ \ There are some generative model doing interpolation , like GANs. Does it helps?\n\ 2. How does CSM compare to Mixup in terms of training efficiency and memory usage?" - source_sentence: llm uncertainty curvature control sentences: - "title: VARIATIONAL QUANTUM ALGORITHMS ARE LIPSCHITZ SMOOTH\n\nsummary: This paper\ \ presents a quite rigorous theoretical analysis of the L-smoothness property\ \ of VQA objective functions which scales linearly on the # of layers P, providing\ \ a worst-case upper limit on curvature that holds for general circuits. The authors\ \ provide a formal proof of global L-smoothness and derive an explicit upper bound\ \ on the smoothness constant L. Furthermore, they show that for certainc classes\ \ of VQAs this bound may take a very simple form, all the way down to be proportional\ \ to the depth of the underlying circuit. This result is then connected to circuits\ \ often assumed to be relevant for practical applications, including a diagnostic\ \ for ansatz overparameterization and a heuristic for setting near-optimal learning\ \ rates. While the contributions are welcome and well-supported, the analysis\ \ is confined to an idealized, noiseless setting, which limits the direct applicability\ \ of its conclusions to contemporary NISQ hardware.\n\nThis paper maybe useful\ \ since it can help better establish, for example, learning rates. $L$, provides\ \ an upper bound on the curvature and guarantees that the landscape is not infinitely\ \ \"spiky and being able to guarantee this it is crucial for gradient-based methods\ \ because it ensures stability since if I know the maximum curvature, I can choose\ \ a learning rate small enough ($\\eta \\approx 1/L$) to guarantee that the optimization\ \ steps will not wildly overshoot a minimum.\n\nHowever, to my view, this paper\ \ does not solve any of the ever present issues of VQAs. While this paper provides\ \ a valuable formalization of L-smoothness with the potential L-informed learning\ \ rate similar to many classical ML problems, the more fundamental and unresolved\ \ problem for VQAs is the lack of a meaningful lower bound on curvature, not the\ \ upper one, a condition that manifests as the barren plateau problem where vanishing\ \ gradients render optimization intractable regardless of the landscape's theoretical\ \ smoothness.\n\nweaknesses and questions: (1) The entire analysis is done in\ \ an idealized noiseless setting. The authors do acknowledges this by establishing\ \ the result as a theoretical baseline. However, this is a significant limitation.\ \ The primary challenge in practical VQA optimization stems from the stochastic\ \ nature of the objective function landscape induced by shot noise and hardware\ \ errors of all shorts. An analysis of L-smoothness in a setting where these dominant,\ \ non-smoothness-inducing effects are absent provides limited guidance for optimization\ \ on actual NISQ devices. The conclusions about stable, predictable curvature\ \ scaling may not hold when the optimizer interacts with a stochastic estimator\ \ of the objective function.\n\n(2) The bound is potentially loose since the proof\ \ of Theorem 2 relies on the inequality $\\|H\\|_2 \\leq\\|B\\|_2$, where $B_{k\ \ l}=4\\|M\\|_2\\left\\|G_k\\right\\|_2\\left\\|G_l\\right\\|_2$ is an element-wise\ \ upper bound on the Hessian matrix $H$. This step can introduce a substantial\ \ gap. The paper's own empirical results as shwon in Figure 1a show that the measured\ \ maximum curvature, $\\tilde{L}_{\\text {max }}$, is often only a small fraction\ \ of the theoretical upper bound $L_{\\text {upper }}$. So while the bound correctly\ \ captures scaling, its significant looseness warrants a more detailed theoretical\ \ investigation maybe. The analysis could be strengthened by discussing the conditions\ \ under which the inequalities in the proof become equalities and what circuit\ \ physical properties (entanglement structure, parameter correlations) might govern\ \ the magnitude of this gap.\n\n(3) The empirical ground truth for maximum curvature,\ \ $\\tilde{L}_{\\text {max }}$, is estimated by taking the maximum Hessian norm\ \ over 1000 random parameter samples. While Appendix D. 2 provides a reasonable\ \ justification for the stability of this estimate, this methodology cannot guarantee\ \ that the true global maximum of $\\left\\|\\nabla^2 f(\\theta)\\right\\|_2$\ \ has been found in general. For that problems where the global optimizer is known\ \ are useful testbeds since hiigh-dimensional landscapes may contain rare and\ \ isolated regions of extreme curvature that are unlikely to be captured by uniform\ \ random sampling. \n\n(4) The proposed heuristic is designed to set a single\ \ global learning rate. However, modern optimization heavily relies on adaptive\ \ methods like adam. So, while the existence of such a constant is proven, this\ \ framework is somewhat misaligned with the reality of modern, large-scale optimization\ \ unless we want to restrict ourselves to only talk about quantum optimization\ \ in isolation. As noted in the literature, e.g. https://arxiv.org/abs/2210.02418\ \ for many typical problems, objective functions rarely satisfy uniform smoothness\ \ assumptions in a way that is practically useful their gradients may only be\ \ locally Lipschitz continuous, or the local curvature can vary dramatically across\ \ the parameter space. Of course, the VQA objective is usually globally L-bounded,\ \ as shown in this paper. But a global constant $L$, determined by the region\ \ of maximum curvature is excessively conservative for the majority of the landscape\ \ as far as using it for thelearning rate. Standard gradient descent with a step\ \ size derived from this global $L$ (e.g., $\\eta \\approx 1/L$) would take impractically\ \ small steps thus leading to slow convergence. This is precisely why SOTA optimizers\ \ really care to account for local geometry. The paper's proposed learning rate\ \ heuristic, while nice in principle, still provides a global rate, which does\ \ not align with modern optimization paradigms. The analysis would be significantly\ \ strengthened by contextualizing its findings within more modern frameworks,\ \ such as local or relative smoothness of the VQA objective in this sense.\n(1)\ \ How do you expect the main results and particularly the predictable linear scaling\ \ of curvature with depth, to change in the presence of realistic shot noise and\ \ hardware noise? This is super crucial. Does the concept of L-smoothness remain\ \ a useful descriptor for the stochastic objective function that an optimizer\ \ actually interacts with?\n\n(2) Could you provide more theoretical insight into\ \ the large gap between the derived upper bound $L_{\\text {upper }}$ and the\ \ empirically observed $\\tilde{L}_{\\text {max }}$ ? Does this gap depend on\ \ properties not captured by the bound, such as the circuit's entanglement capacity\ \ or the locality of the observable?\n\n(3) The trigonometric polynomial proof\ \ route in Appendix A. 6 bounds the Fourier coefficients as $\\left|d_\\omega\\\ right| \\leq\\|M\\|_2$. Given that these coefficients have a specific structure\ \ ( $d_\\omega=\\left\\langle u_\\omega\\right| M\\left|v_\\omega\\right\\rangle$\ \ ), could a more refined analysis that does not resort to this uniform worst-case\ \ bound yield a tighter overall smoothness constant? These trigonometric polynomials,\ \ note, are actually Hermitian trigonometric polynomials in $d$ complex variables\ \ and the optimization takes place over the torus $\\mathbb{T}^d$. does this not\ \ induce some \"structure\" to be exploited so as to further bound $L$? \n\n(4)\ \ Regarding the learning rate heuristic, would it be more effective to use the\ \ calibrated effective smoothness constant, $L_{\\rm effective}$, to rescale\ \ the global learning rate of an adaptive optimizer like adam, rather than using\ \ it directly in a vanilla SGD context?" - "title: Text Has Curvature\n\nsummary: The paper is dedicated to the question\ \ of intrinsic curvature of discrete textual data and asks, whether we could define\ \ a practical measure for its estimation? The authors claim that curvature should\ \ be a property of string data itself rather than an artifact of specific embedding\ \ space (hyperbolic, spherical etc.) and propose \"Texture,\" a measure of curvature\ \ primitive that measures how left and right contexts interact in-context. The\ \ authors claim three contributions: (1) empirical and theoretical justification,\ \ proving that semantic inference in natural corpora is non-flat; (2) a formal\ \ definition of Texture as a signed curvature field via optimal transport; and\ \ (3) practical applications for long-context inference via pruning (CURVPRUNE)\ \ and retrieval (CURVFLAG).\n\nweaknesses and questions: Strengths:\n\n- The paper\ \ addresses an important question, that hasn't attracted much attention: what\ \ does curvature mean for text itself, independent of embedding spaces. \n\n-\ \ The theory is rather sound and interesting. The use of holonomy and contextual\ \ evidence interaction (CEI) as falsifiable null hypotheses is methodologically\ \ strong. By testing against coherence-destroying controls (suffix-swap, local-shuffle),\ \ the authors provide empirical evidence that an approach is theoretically motivated.\n\ \n- The paper shows interesting applications of their curvature estimates with\ \ some performance improvements.\n\nWeaknesses:\n\n- The paper claims are somewhat\ \ misleading. Initially they state that their curvature measure must be independent\ \ from the target model's choice of embedding space. Although it's independent\ \ from the target model the approach is evaluated at, it appears to be dependent\ \ on the choice of bidirectional encoding/infilling model, isn't it? ($\\textit{distilroberta-base}$\ \ in this case)\n\n- While the paper claims to establish discrete curvature at\ \ the word level in the abstract, the notation is somewhat ambiguous with later\ \ claims being made for token-level representations. Moreover, although authors\ \ claim to take a union of Top-K tokens with an intention to preserve the support\ \ -- however, I didn't find any discussion related to the choice of K to be used.\ \ The use of character-level / byte-level models for their approach looks rather\ \ more sound [1, 2]. \n\n\n- The computational cost of computing Texture (solving\ \ Sinkhorn scaling per slot) may be prohibitive for very long contexts. The paper\ \ does not provide a thorough cost-benefit analysis.\n\n- The paper lacks extensive\ \ ablation on the choice of key hyperparameters (support size K; context radii\ \ L, R; infilling model) and their effects on curvature sign and magnitude. All\ \ the experiments are limited to Llama-3-8B.\n\n\nThe appendices provides open-sourced\ \ code samples for reproduction, algorithmic details, and additional experiments.\n\ Although the topic is interesting, I think the paper would benefit from additional\ \ revision and discussion of tokenization biases and their impact on curvature\ \ estimates and experimental evaluation with ablations on the choice of hyperparameters\ \ and the sensitivity of their approach to these choises. Overall, I believe the\ \ work is still quite immature and is not yet ready for publication.\n\n\n[1]\ \ Juan Luis Gastaldi, John Terilla, Luca Malagutti, Brian DuSell, Tim Vieira,\ \ & Ryan Cotterell (2025). The Foundations of Tokenization: Statistical and Computational\ \ Concerns. In The Thirteenth International Conference on Learning Representations,\ \ ICLR 2025, Singapore, April 24-28, 2025.\n\n[2] Jonathan Hayase, Alisa Liu,\ \ Noah A. Smith, & Sewoong Oh (2025). Sampling from Your Language Model One Byte\ \ at a Time. arXiv preprint arXiv: 2506.14123.\n- Is curvature dominated by local-interactions\ \ only, i.e. we should primarilly consider small L and R?\n\n- How is $s_{ref}$\ \ chosen in Eq.(2)? Is the condition in Certificate I necessary or sufficient?" - "title: Debiasing Mini-Batch Quadratics for Applications in Deep Learning\n\n\ summary: This paper demonstrates a systematic bias in stochastic 2nd order approximation\ \ of empirical risk for neural networks. The study start from an observation showing\ \ that a particular curvature structure on minibatch Hessian is biased towards\ \ underestimation. Then, authors proposes a debiasing method.\n\nweaknesses and\ \ questions: I believe the notion of unbiased estimate is not well defined and\ \ studied. Define the Hessian associate with full-batch training loss as $$H(x)\ \ = \\frac{1}{n} \\sum_{i=1}^n \\nabla^2 f_i(x)$$. Similarly, we define the unbiased\ \ estimate of this matrix as $$H'(x) = \\frac{1}{m} \\sum_{k=1}^m \\nabla^2 f_{i_k}(x)$$\ \ where $i_k$ are uniformly drawn from $\\{1,\\dots, n\\}$. \nFor each fixed vector\ \ $d$, we have $$E d^\\top H' d^\\top = d^\\top H d$$. Thus, the **directional\ \ curvature**, defined in the paper, is unbiased. But, why this paper observe\ \ a bias in the estimate? The main issue is that they consider random directions\ \ $d$ depending on minibatches. I do not know why we need to choose random $d$s?\ \ \n\nI believe the quantity of interest is not well defined and motivated, here.\ \ It is easy to provide an unbiased estimation of $q$, directional curvature\ \ and directional slop. But, the authors want to estimating another quality that\ \ they do not exactly defined and it is not clear how its estimation connect with\ \ Newton's method or Laplace approximation. \n\nIf you want to estimate the maximum/minimum\ \ eigenvalue of the Hessian or even its condition number, the empirical Hessian\ \ provides an asymptotically unbiased estimate (see for example https://arxiv.org/pdf/1912.10754).\n\ - I recommend to replace $u_1$ and $u_2$ in Figure 1 by leading eigenvectors of\ \ full-batch Hessian to better grasp my comment in weaknesses. I expect to see\ \ that increasing batch size will lead to a better approximation of the curvature.\ \ \n- What is exactly the quantity that you want to estimate and why it is important\ \ to estimate? Directional curvature on which random directions?\n- What do you\ \ exactly mean by bias? To prove an estimate has bias, we need to take average.\ \ But, experiments in Figure 1 are not computing an average. How can I conclude\ \ from these experiments that the estimate is biased?" - "title: What's the Magic Word? A Control Theory of LLM Prompting\n\nsummary: This\ \ work presents a control perspective of the LLM steerability by introducing the\ \ concept of $k-\\epsilon$ controllability. They also proved a bound on the controllability\ \ of self-attention in terms of the singular values of its weight matrices. In\ \ addition, several experimental studies have performed to compute the $k-\\epsilon$\ \ controllability of LLMs (Falcon-7b, Llama-7b, Falcon-40b). The authors conclude\ \ that LLMs are very controllable and the control prompts of 10 tokens or less\ \ are sufficiently enough to ensure the LLM output the target token.\n\nweaknesses\ \ and questions: Section 4 only considers a self-attention head, which is quite\ \ simple and limited (compared to the current model used in LLM). What are the\ \ difficulties in generalizing such results to a more complex model?\n\nThe presentation\ \ of Section 4 can be further improved. The relationship between state controllability\ \ (Definition 7) and $k-\\epsilon$ controllability (Definition 6) should be discussed,\ \ i.e., implications of your theory result in Section 4. In addition, I am confused\ \ about some notations: are $u_i$, $x_i$ the embeddings of the tokens? Previously\ \ the u and x are presented as tokens, it does not make sense to make $\\|u_i\\\ |\\le 1$ and $\\|x_i\\| \\le 1$ if they are tokens. In addition, is this assumption\ \ valid in real LLMs? \n\n\nAlthough introducing the controllability of LLM from\ \ a control perspective is interesting, the experimental results of checking the\ \ controllability of the LLMs are not very exciting given the existing results\ \ from previous work [Zou 2023]. The experiment setup is almost identical to GCG\ \ work and the obtained results are also within expectation. Instead, proposing\ \ a new method to study the controllability of black-box LLMs will be more interesting.\n\ 1. What are the connections between state controllability (Definition 7) and $k-\\\ epsilon$ controllability (Definition 6)? Does the former imply the later?\n2.\ \ Are the assumptions $\\| u_i\\| \\le 1$ and $\\| x_i \\| \\le 1$ realistic?\ \ If not, is your results in Section 4 still hold?" - "title: Uncertainty-Aware LLM Probing\n\nsummary: The authors examine how effective\ \ uncertainty quantification methods are at quantifying the uncertainty of LM\ \ probes, and design their own gradient-based quantification method. They find\ \ that no method is robust to domain shifts, but that their method can detect\ \ OOD effectively.\n\nweaknesses and questions: 1. The writing overall tends to\ \ be unclear and not well-organized. Many sentences lack specificity or need to\ \ be better defined. (ex. Line 58 “turns out to be sub-optimal”—how? Lines 75-77,\ \ “Based on evaluations on uncertainty quantification in traditional ML, we would\ \ expect the uncertainty methods to work reliably here”—which evaluations, and\ \ why?)\n2. Some of the results would benefit from further analysis, as inconclusive\ \ results are reported without investigation into the cause. For instance, lines\ \ 313-314, “More precisely, we sometimes observed very high performance, but very\ \ low one at other times”, does not present any hypothesis for why this was the\ \ case. \n3. The performance benefit on ID data does not appear to be consistent\ \ and significant. It only outperforms all other methods in “average rank” with\ \ 2/5 models. Figure 2 also suggests to me that these metrics all achieve similar\ \ scores, which does not present a compelling case for using ADGrad over other\ \ established methods.\n4. While OOD data detection is useful, it is not immediately\ \ obvious why this should be compared to uncertainty quantification methods as\ \ baselines and not baselines for OOD data detection. This seems like a separate\ \ application.\nWhy report the average rank of the scores and not simply average\ \ over all categories? This seems somewhat arbitrary." - 'title: Representational Curvature Modulates Behavioral Uncertainty in Large Language Models summary: The paper argues that straighter hidden-state trajectories in LLMs are associated with lower next-token uncertainty. It shows that this curvature–entropy link is strongest in middle layers, emerges during training, and can be partly controlled with perturbations or regularization, suggesting that trajectory straightening may play a functional role in prediction rather than being just a descriptive pattern. weaknesses and questions: **Strengths:

 ** 1. This paper checks where the studied effect is strongest, how it emerges over training, and whether trajectory-aligned perturbations selectively change entropy while misaligned perturbations do not. That makes the mechanistic claim meaningfully stronger than a simple observational result. **Weaknesses:** 1. The main variables in the study are scalars, ie, contextual curvature, computed over a local trajectory window, and next-token entropy. That makes the story elegant, but also lossy: very different geometric structures can share similar curvature, and very different output distributions can share similar entropy. So the paper may be capturing a real effect while still missing the richer mechanism underneath it. 2. The regularization experiments are done on small-scale LLMs, and the paper says outright that it remains untested whether the same relationship holds for larger foundation models, different architectures, multimodal systems, or more heterogeneous datasets. That matters a lot here, because the paper is close to proposing a general principle of autoregressive prediction. Right now, I think the evidence is good for “this happens in the studied setting,” but weaker for “this is a broad law of LLM computation.” 3. Even if curvature regularization reliably lowers entropy, the paper does not show that this improves calibration, robustness, task accuracy, or downstream usefulness. And because validation loss stays similar, one interpretation is that it mostly reshapes probability mass among wrong answers rather than improving decision quality. * The regularization result makes curvature look less like a correlate of entropy and more like a direct control variable. If entropy changes while validation loss stays the same, why shouldn’t we read this as evidence that curvature is effectively determining uncertainty? * The observation that middle layers are both straightest and most predictive of entropy sounds reminiscent of an information-bottleneck story, where intermediate representations compress context into a task-relevant predictive code. Do you see your result that way, or do you think “trajectory straightening” is a different phenomenon from compression in the information-bottleneck sense?' - 'title: Conf-Gen: Conformal Uncertainty Quantification for Generative Models summary: The authors present an extension of conformal risk control (CRC) which is itself an extension of conformal prediction (CP), suited to obtaining performance guarantees for the answers provided by LLMs, in the case where there is a set of exchangeable (input, ground_truth) pairs. One has to also interpret the LLM output to form a set or sequence of candidate answers parametrized by a conservativenss parameter lambda (e.g., to select the most promising candidates) and provide both a way to obtain a score for each candidate answer and a way to evaluate how good this output is given a ground truth answer. A calibration method then allows to choose lambda to obtain a required degree of expected goodness on a new example from the same distribution. weaknesses and questions: Soundness and originality This paper allows doing a form of calibration with conformal guarantees on LLM outputs in settings not considered before, which makes comparisons difficult in several of these settings. The experiments suggest that the theoretical guarantees hold. The paper appears to be very sound and original. Presentation The paper does a good job of presenting relevant prior work. The formal aspect of the paper also appears much above the ICML average and I did not detect any issue on that front, although I did not read the appendix. More generally, everything is very clear. Significance This extension of conformal prediction seems relevant for many LLM-related tasks and could thus be impactful. See the key questions for authors. One limitation I would note is that the kind of data stream on which LLMs are actually applied is not necessarily exchangeable, invalidating the core assumption behind CP. But the guarantees may be useful in practice nonetheless, e.g., to obtain conservative safety guarantees. Another CP limitation of course is that these are only guarantees in average, not for a particular query. This is of course not a specific issue with this paper, but it may limit the usefulness of this family of approach, e.g., in high-stakes domains like medical decision-making. Please share any thoughts on these two issues (non-exchangeability and average-guarantees).' - "title: Local Curvature Descent: Squeezing More Curvature out of Standard and\ \ Polyak Gradient Descent\n\nsummary: Convexity and L-smoothness are standard\ \ assumptions in optimization literature which are useful for easier analysis\ \ of optimization algorithms and determining the right algorithm parameters. These\ \ global conditions may not always take local differences in the curvature into\ \ account. This paper proposes new analogues of the assumptions to incorporate\ \ certain kinds of local curvature information. The paper also proposes modifications\ \ of gradient descent using matrix valued step sizes to take advantage of the\ \ modified assumptions.\n\nweaknesses and questions: The first-order convexity\ \ condition and L-smoothness imply that $$ f(y) + \\langle \\nabla f(y), x-y\\\ rangle \\leq f(x) \\leq f(y) + \\langle \\nabla f(y), x-y\\rangle + \\frac L2\ \ || x-y||^2.$$ The authors' assumption 2.1 modifies these inequalities by adding\ \ the term $\\frac12 || x-y||^2_{\\mathbf C}$ to the lower bound and upper bound\ \ for $f(x)$ provided by these inequalities (since $\\frac12 || x-y||^2_{\\mathbf\ \ C+L\\mathbf I} = \\frac12 || x-y||^2_{\\mathbf C} + \\frac L2 || x-y||^2$).\ \ Thus, assumption 2.1 seems to be more general than L-smoothness but it is less\ \ general than convexity.\n\nIn the standard analyses of GD, L-smoothness guarantees\ \ a sufficient decrease with each step (with the right step size) and convexity\ \ ensures that that decrease pushes us towards the minimizer. The two inequalities\ \ balance each other in a crucial way. This work exploits that tradeoff. However,\ \ it is not surprising that if the same term is added to both the lower bound\ \ and upper bound of $f(x)$ then they will cancel each other out and the standard\ \ convergence proofs will still go through. \n\nFurthermore, if $f$ is twice differentiable,\ \ assumption 2.1 is actually equivalent to\n$$\\mathbf C(x) \\preceq \\nabla^2\ \ f(x) \\preceq \\mathbf C(x) + L_C \\mathbf I.$$ This follows from the same kind\ \ of standard arguments used to show that convex functions have positive semidefinite\ \ Hessians. Having observed this second order condition, many of the remarks that\ \ the authors make follow directly. The case when $\\mathbf C = 0$ is the standard\ \ case with convexity and L-smoothness, and the case when $L_C=0$ is the realm\ \ of second order methods like Newton's method. My impression is that to give\ \ any useful advantage over standard GD, the map $\\mathbf C(x)$ will have to\ \ approximate the Hessian $\\nabla^2 f(x)$. But then the algorithms provided do\ \ not seem to be very useful unless there is a good way to approximate the Hessian.\n\ \nLCD1 actually just seems to be a version of Newton's method where the Hessian\ \ is overestimated by its upper bound $\\mathbf C(x) + L_C \\mathbf I$ to make\ \ it more stable. The convergence rate provided for LCD1 is the same as GD (unless\ \ $L_C = 0$, in which case it is pure Newton's method anyway), which does not\ \ provide any new insights either. The authors present LCD2 as a generalization\ \ of Polyak's step size, but the step size $\\beta_k$ in that case does not even\ \ have a closed form. Computing $\\beta_k$ itself requires an optimization problem\ \ to be solved at each step, and the benefits of doing that are not clear. LCD3\ \ has a closed form step-size but there are no convergence results provided for\ \ it, so it's not clear how well it performs.\n\nThe assumption could still have\ \ been justified with examples of interesting functions that satisfy assumption\ \ 2.1 in non-trivial ways. Unfortunately, that does not seem to be the case. One\ \ of the curvature matrices specified for each of the examples 6.1-6.4 is just\ \ the Hessian. For examples 6.2 and 6.4, $\\nabla f(x) \\nabla f(x)^\\top$ is\ \ proposed as another candidate for the curvature matrix, but these kinds of approximations\ \ of the Hessian are already covered by quasi-Newton methods like Berndt–Hall–Hall–Hausman\ \ algorithm.\n\nThe experiments are also only performed on these trivial examples\ \ and authors compare their proposed step sizes only against Polyak step size.\ \ The first experiment is on a strongly convex and L-smooth function, which is\ \ covered by the classical assumptions, and the optimal method for which would\ \ have been a momentum-based algorithm like Nesterov's accelerated gradient descent.\ \ The second experiment chooses the Hessian as the curvature matrix, reducing\ \ it to the case where second order methods would perform better.\n\nOverall,\ \ the assumption proposed in the paper does to offer many new theoretical insights\ \ nor do the algorithms proposed offer practical advantages over existing algorithms.\n\ - Can the authors comment on the second order characterization of assumption 2.1?\ \ \n- What is the time complexity of LCD2 compared to Polyak step size? Specifically,\ \ how does the computation of $\\beta_k$ affect the complexity?" - source_sentence: theory on finite-trial estimates sentences: - 'title: Complexity Analysis of Normalizing Constant Estimation: from Jarzynski Equality to Annealed Importance Sampling and beyond summary: This paper provides a full theoretical analysis of the error in Annealed Importance Sampling (AIS), accounting for both the sampling process that generates particles and the estimator of the normalizing constant computed from these samples. The authors derive a general upper bound on the estimation error as a function of the prescribed probability path that guides the sampling process. Two important path choices are examined: the standard geometric interpolation path, which is shown to lead to exponential complexity in the difficulty of the problem, and the reverse diffusion path, which achieves only polynomial complexity—provided oracle access to score functions (otherwise, an additional approximation error appears). The difficulty of the problem is quantified in terms of the between-mode distance of the target distribution. weaknesses and questions: The writing could be clearer in some parts, but overall the paper is clear. Q1. Theorem 4 is particular to the geometric interpolation path? Is it normal that the number of samples $N$, the number of SMC iterations $M$, and the discretization of the Langevin process, do not appear in the error bound (Eq 11)?' - 'title: Stronger Neyman Regret Guarantees for Adaptive Experimental Design summary: This paper explores efficient ATE estimation in adaptive experimental designs. The authors focus on Neyman regret, which quantifies the variance difference between the inverse-propensity-weighted (IPW) estimator under the proposed adaptive design and the best fixed design in hindsight. Prior work (e.g., Dai et al., 2023) established a sublinear $O(\sqrt{T})$ bound on Neyman regret. This paper strengthens that result, achieving an $O(\log T)$ bound under slightly stronger assumptions. The analysis is further extended to contextual (multigroup) settings, introducing a method that ensures $O(\sqrt{T})$ regret across multiple overlapping subpopulations. The approach is validated both theoretically and empirically. weaknesses and questions: None. None.' - "title: Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional\ \ Samplers\n\nsummary: This paper provides a non-asymptotic convergence theory\ \ for diffusion models with a mismatched score function. The results can be applied\ \ to zero-shot conditional samplers, especially linear conditional models. The\ \ authors also propose the Bias-Optimal (BO) DDNM sampler and provide a comparison\ \ with the previous algorithms.\n\nweaknesses and questions: - Assumption 2 requires\ \ an upper bound on the score estimation error $\\epsilon^2 = \\tilde{\\mathcal{O}}(T^{-2})$,\ \ which could be restrictive compared to the previous literature (e.g., Li et\ \ al. 2024b) which applies to any $\\epsilon$. \n- The in-line equations in Section\ \ 2.1 and Section 5 are hard to follow. I suggest the authors re-organize the\ \ equations for better readability, especially by highlighting the definitions\ \ and differences of $q$, $p$, and $\\hat{p}$.\n- The big-O notation in this work\ \ is confusing. To name a few, in Assumptions 3 and 4, should $(1-\\alpha_t)^m\ \ \\mathbb{E}[\\cdot] = O((1-\\alpha_t)^m)$ be equivalent to $\\mathbb{E}[\\cdot]\ \ = O(1)$? Or do the hidden constants in these assumptions implicitly depend on\ \ $(1-\\alpha_t)^m$? In Definition 1, the noise schedule needs to satisfy $\\\ bar{\\alpha}\\_T = o (1/T)$ which is defined as an asymptotic bound $\\lim\\sup_{T\\\ to\\infty} |\\bar{\\alpha}_T / (1/T)| \\to 0$, while Theorem 1 presents a non-asymptotic\ \ analysis. How does the asymptotic assumption apply to the non-asymptotic analysis?\ \ I suggest the authors clarify the notations or even explicitly write the constants'\ \ dependency on the hyperparameters if possible. \n- Does $1-\\alpha_1=\\delta$\ \ in equation (8) contradict with $1-\\alpha_1 \\lesssim \\frac{\\log T}{T}$ in\ \ Definition 1? What can we obtain from Theorem 1 if $\\alpha_t$ is chosen as\ \ (8)?" - 'title: The Number of Trials Matters in Infinite-Horizon General-Utility Markov Decision Processes summary: The paper analyzes the impact of the number of trails in estimating the objectives for GUMDPs. For both the discounted and average settings, it is shown by examples that there are mismatches between the finite-trial estimates and the actual infinite-trail objectives. Bounds on the mismatches are provided, with numerical results supporting the theoretical claims. weaknesses and questions: None In Fig 3b, it looks like there are some discontinuities in the performance of $M_{f, 3}$ around $\gamma=0.9$ where the finite-trail performance seems to diverge away from the infinite-trail one, but then converges back to it. Is that expected from theoretical analysis?' - "title: The Minimax Rate of HSIC Estimation for Translation-Invariant Kernels\n\ \nsummary: The rate at which HSIC can be estimated is an important and open problem,\ \ in this paper, the authors prove that\nthe minimax optimal rate of HSIC estimation\ \ for Borel measures is $\\mathcal{O}(n^{-0.5})$ with M>=2 components, which is\ \ very important as existing conclusion only holds for M=2. Other byproducts can\ \ be naturally introduced, implying the minimax lower bound for the estimation\ \ of\ncross-covariance operator, which can be further specialized to get back\ \ the minimax result on the estimation of the covariance operator.\n\nweaknesses\ \ and questions: 1. Overall, the paper is not easy to follow as the paper's main\ \ contribution seems to be the proof part. \n2. I wouldn't say it is the weakness\ \ or the author's problem, as this is a theoretical paper, experiments are not\ \ necessary. Still is it possible to design toy experiments to validate the conclusions\ \ in the paper?\nNA" - 'title: Active Treatment Effect Estimation via Limited Samples summary: Experimental design for estimating treatment effects does not generally have strong finite-sample guarantees, especially as the dimensionality of the covariates grows. Recent works implement experimental design based on leverage scores. This work proposes an alternative approach called IRD, which helps achieve a sample complexity for the estimation error that is linear in the covariate dimensionality. The method is validated with a variety of standard semi-synthetic experiments. **Update after rebuttal**: after considering the additional results provided, I have decided to increase my score. weaknesses and questions: The problem of active sampling for treatment-effect estimation with high-dimensional covariates is clearly significant. The solution appears to have clear benefits over other recent works. It would be very helpful to better describe the method so that readers can understand the key contributions. 1. Specifically what role do partitioning and subsampling play in the proposed method? 2. Does this method easily extend to multiple treatments?' - "title: Beyond the Average: Distributional Causal Inference under Imperfect Compliance\n\ \nsummary: This paper introduces an estimator for the local distributional treatment\ \ effect, which is an instrumental-variable-type estimator for the causal effect\ \ among compliers. They prove asymptotic properties of the estimator and empirically\ \ argue using synthetic and real data that the estimator is more effective than\ \ baselines.\n\nweaknesses and questions: Strengths:\n- while I do have some confusions\ \ outlined below, I think the contribution here is probably theoretically solid\ \ and well-scoped\n- experiments are mostly clear and compelling\n\nWeaknesses:\n\ - could use some more clarity on exactly what the contribution here is - is it\ \ just adding a distributional component onto existing IV estimators for complier\ \ effects? or something further. I think it's more but have trouble totally figuring\ \ it out, the distributional component by itself seems like probably not quite\ \ enough novelty but I could be wrong. Lemma 3.2 would be a nice place to do some\ \ of this clarification but anywhere is fine\n- confused about a couple fundamental\ \ pieces here: the authors define \\eta_z = E[D | Z, S, X] on L193. However according\ \ to the causal graph in Fig 1 it looks like D should be independent of X, S given\ \ Z - why is this not just E[D | Z] in that case? Similarly I wonder about the\ \ equation below on L194 - I would assume from Fig 1 that Y should be independent\ \ of X, S given Z\n- Algo 1: should clarify what exactly M is trained on, inputs\ \ + outputs. I don't see this notation elsewhere\n- Assumption 5.1: (I) I find\ \ it confusing why we need a condition on the difference but not on the individual\ \ components, (ii) the < \\infty part of this is confusing to me as well, doesn't\ \ this hold trivially for any F bounded above by 1? which \\mu_z should be since\ \ it is an Expectation of an indicator function\n- Theorem 5.2 - I may just not\ \ have the theory background here so apologies but I don't really understand what\ \ it means for this to converge to a GP. Don't we want it to be decreasing in\ \ some function of n?\n- Fig 4: your left and right plots look identical to me,\ \ I'm assuming this is an error\n- clarify the contribution beyond previous work\ \ \n- correct Fig 4 so that the L and R parts are different\n- clarify how I should\ \ interpret causal graph in Fig 1 and how it interacts with the definitions of\ \ \\eta and \\mu" - "title: Distributed Least Squares in Small Space via Sketching and Bias Reduction\n\ \nsummary: Sketched least squares involve estimating the term $(X^TX)^{-1}$ which\ \ has a high bias when the sketch matrix $S$ is not sub-Gaussian. This paper gives\ \ a sparse sketching method using a LESS embedding which runs in optimal space\ \ and current matrix multiplication time, where $S$ is sparse, and constructed\ \ based on the leverage scores of the data matrix $A$ (Definition 2 $(s,\\beta_1,\\\ beta_2)$-LESS embedding). The paper also improves the sharpness of the probability\ \ bounds which is applicable to similar problems in RMT using LESS embeddings.\ \ For $s=1$, nothing is different, but when $s > 1$, the bias bound is reduced.\n\ \nweaknesses and questions: 1. It would be nice to have experiments with other\ \ sketching methods to (empirically) justify some statements on the bias, variance\ \ (although not stated) and computational time, e.g. estimators mentioned in Table\ \ 1. For example, are there sketching estimators with higher bias, but less variance?\ \ \n\nThere are some minor typos, e.g. line 212 reference missing, lines 282 to\ \ Equation (2) at bottom of the page is missing a bracket for the numerous expectations\ \ (in contrast to lines 542 onwards in the appendix).\n\nIn Appendix A, notation\ \ for concentration inequalities should be looked at and made consistent, e.g.\ \ Lemma 6 / H{\\\"o}lder's inequality should have a $\\frac{1}{q}$, Lemma 7 ($\\\ lambda$ max isn't defined), Lemma 8 / Azuma's inequality ($\\lambda$, $m$ should\ \ be consistent), Lemma 10 is missing a bracket for $\\mathbb E[x_i^2]$. \n\n\ The presentation of the proof for Theorem 5 was slightly jarring (due to Lemma\ \ 11, Lemma 12 appearing in the proof), but there also doesn't seem to be a good\ \ way to include them (since referring to the two lemmas requires the upper bounds,\ \ and flipping back a page is also inconvenient). Maybe a solution is to indent\ \ the Lemmas, or box them up?\n\nThe dot before line 626 (after 72) should be\ \ removed.\n1. It would be nice to have experiments with other sketching methods\ \ to (empirically) justify some statements on the bias, variance (although not\ \ stated) and computational time, e.g. estimators mentioned in Table 1. For example,\ \ are there sketching estimators with higher bias, but less variance? \n\n2. Despite\ \ being clear to read, I had to go back and forth a bit to find out what the novelty\ \ is. I appreciate the clarity and thoroughness of explaining the bounds on the\ \ bias and variance, runtime, but I would like it if Definition 1 & 2 came much\ \ earlier (or at least maybe an informal Definition 2 after line 85?)" pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@10 - cosine_precision@10 - cosine_precision@100 - cosine_recall@10 - cosine_recall@100 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 model-index: - name: SentenceTransformer results: - task: type: information-retrieval name: Information Retrieval dataset: name: reviewsearch type: reviewsearch metrics: - type: cosine_accuracy@1 value: 0.35884636716583473 name: Cosine Accuracy@1 - type: cosine_accuracy@10 value: 0.7750214289315787 name: Cosine Accuracy@10 - type: cosine_precision@10 value: 0.19142338526697927 name: Cosine Precision@10 - type: cosine_precision@100 value: 0.05602279029899663 name: Cosine Precision@100 - type: cosine_recall@10 value: 0.17017869143702086 name: Cosine Recall@10 - type: cosine_recall@100 value: 0.4395403885775364 name: Cosine Recall@100 - type: cosine_ndcg@10 value: 0.25119901516426524 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4888785045607057 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.15688999637818096 name: Cosine Map@100 --- # SentenceTransformer This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Maximum Sequence Length:** 2048 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Cosine Similarity - **Supported Modality:** Text ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'unpad_inputs': False, 'architecture': 'Gemma3TextModel'}) (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True}) (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'}) (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'}) (4): Normalize({}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference queries = [ 'theory on finite-trial estimates', ] documents = [ 'title: The Number of Trials Matters in Infinite-Horizon General-Utility Markov Decision Processes\n\nsummary: The paper analyzes the impact of the number of trails in estimating the objectives for GUMDPs. For both the discounted and average settings, it is shown by examples that there are mismatches between the finite-trial estimates and the actual infinite-trail objectives. Bounds on the mismatches are provided, with numerical results supporting the theoretical claims.\n\nweaknesses and questions: None\nIn Fig 3b, it looks like there are some discontinuities in the performance of $M_{f, 3}$ around $\\gamma=0.9$ where the finite-trail performance seems to diverge away from the infinite-trail one, but then converges back to it. Is that expected from theoretical analysis?', "title: The Minimax Rate of HSIC Estimation for Translation-Invariant Kernels\n\nsummary: The rate at which HSIC can be estimated is an important and open problem, in this paper, the authors prove that\nthe minimax optimal rate of HSIC estimation for Borel measures is $\\mathcal{O}(n^{-0.5})$ with M>=2 components, which is very important as existing conclusion only holds for M=2. Other byproducts can be naturally introduced, implying the minimax lower bound for the estimation of\ncross-covariance operator, which can be further specialized to get back the minimax result on the estimation of the covariance operator.\n\nweaknesses and questions: 1. Overall, the paper is not easy to follow as the paper's main contribution seems to be the proof part. \n2. I wouldn't say it is the weakness or the author's problem, as this is a theoretical paper, experiments are not necessary. Still is it possible to design toy experiments to validate the conclusions in the paper?\nNA", 'title: Stronger Neyman Regret Guarantees for Adaptive Experimental Design\n\nsummary: This paper explores efficient ATE estimation in adaptive experimental designs. The authors focus on Neyman regret, which quantifies the variance difference between the inverse-propensity-weighted (IPW) estimator under the proposed adaptive design and the best fixed design in hindsight. Prior work (e.g., Dai et al., 2023) established a sublinear $O(\\sqrt{T})$ bound on Neyman regret. This paper strengthens that result, achieving an $O(\\log T)$ bound under slightly stronger assumptions. The analysis is further extended to contextual (multigroup) settings, introducing a method that ensures $O(\\sqrt{T})$ regret across multiple overlapping subpopulations. The approach is validated both theoretically and empirically.\n\nweaknesses and questions: None.\nNone.', ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) print(query_embeddings.shape, document_embeddings.shape) # [1, 768] [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) # tensor([[0.5815, 0.5067, 0.4693]]) ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `reviewsearch` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.InformationRetrievalEvaluator) with these parameters: ```json { "query_prompt": "task: search result | query: ", "corpus_prompt": "title: none | text: " } ``` | Metric | Value | |:---------------------|:-----------| | cosine_accuracy@1 | 0.3588 | | cosine_accuracy@10 | 0.775 | | cosine_precision@10 | 0.1914 | | cosine_precision@100 | 0.056 | | cosine_recall@10 | 0.1702 | | cosine_recall@100 | 0.4395 | | **cosine_ndcg@10** | **0.2512** | | cosine_mrr@10 | 0.4889 | | cosine_map@100 | 0.1569 | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 46,935 training samples * Columns: anchor, positive, negative_1, negative_2, negative_3, negative_4, negative_5, negative_6, and negative_7 * Approximate statistics based on the first 100 samples: | | anchor | positive | negative_1 | negative_2 | negative_3 | negative_4 | negative_5 | negative_6 | negative_7 | |:---------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------| | type | string | string | string | string | string | string | string | string | string | | modality | text | text | text | text | text | text | text | text | text | | details | | | | | | | | | | * Samples: | anchor | positive | negative_1 | negative_2 | negative_3 | negative_4 | negative_5 | negative_6 | negative_7 | |:--------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | meta-learning unclear contribution | title: Extending Contextual Self-Modulation: Meta-Learning Across Modalities, Task Dimensionalities, and Data Regimes

summary: This paper proposes to extend Contextual Self-Modulation (CSM), which is an uncertainty-handling mechanism of Neural Context Flow (NCF). NCF is a robust meta-learning framework of neural ODE that includes self-modulation with high-order Taylor expansion around contexts. The authors focus on generalizing CSM into high data regimes and designed StochasticNCF and FlashCAVIA, which take CSM in various directions. From comparative experiments, the author verified that the NCF can be successfully extended to various optimization and meta-learning techniques.

weaknesses and questions: 1. I believe this paper is not clearly written.
* iCSM: It is briefly mentioned that the space of iCSM is "an infinite-dimensional variation" that utilizes "a space of multi-layer perceptrons, whose weights are flattened into a 1-dimensional tensor." I do not find a good enough ex... | title: Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learning

summary: This paper introduces FAME (Fast and Meta knowledge learners), a novel framework for continual reinforcement learning that draws inspiration from the human brain’s memory systems. It proposes a dual-learner architecture comprising a fast learner, which rapidly adapts to new tasks, and a meta learner, which incrementally integrates knowledge to prevent catastrophic forgetting. To support efficient adaptation and stability, the authors define principled metrics such as MDP distance (for task similarity) and catastrophic forgetting (for performance degradation). A key contribution is the adaptive meta warm-up mechanism, which selects the best initialization strategy for new tasks using statistical hypothesis testing, thereby mitigating negative transfer. Theoretical foundations are coupled with practical algorithms for both value-based and policy-based RL, using divergence-based incremental u... | title: Meta-Reinforcement Learning for Compiler Optimization: A Kernel-Embedded CompilerLLM with Verified Assumptions and Practical Guarantees

summary: This paper proposes methods to improve optimization, specifically by trying to find surprising or interesting optimizations that nonetheless pass validation.

weaknesses and questions: 1. This paper is not complete. There are obvious issues with the writing such as "Meta-optimization" and "Meta-reinforcement learning" being repeated many times in the intro, the "Meta-Learning Theory." section in the related work being empty, etc.
2. Clarity could use improvement. In many places (e.g. the abstract) the writing was either to jargon-filled or incomplete for me to understand well.
3. The description in section 4.2 and beyond is not clear enough for me to fully understand the method. There are many places that are unclear, but for instance it is not stated how `k_cfg`, `k_data`, `k_inst` etc. are calculated.
4. There is no comparison with o... | title: Meta-Router: Bridging Gold-standard and Preference-based Evaluations in LLM Routing

summary: This paper addresses the challenge of training LLM routers to balance response quality and inference cost. It proposes a novel causal inference framework, viewing gold-standard and preference-based evaluation data as a treatment assignment problem. The key insight is that bias in preference-based data corresponds to the conditional average treatment effect (CATE). The proposed Meta-Router framework corrects this bias via meta-learners (S-learner, T-learner, X-learner), incorporates propensity score weighting to address data source imbalances, and applies distributionally robust optimization (DRO) for routing robustness. Experiments on benchmarks (AlpacaEval, MT-Bench, MMLU, GSM8K) show improved cost-quality trade-offs compared to standard baselines.

weaknesses and questions: **Limited Baseline Scope**: Compares mostly to classical methods (IPW, DR); recent LLM routing methods and stron... | title: The Meta-Representation Hypothesis

summary: The paper proposes to combine Deep Mutual Learning with RL. In Deep Mutual Learning, several learners learn independently but at the same try to minimize the KL between their predictive distributions. The paper hypothesizes that two RL policies can learn from different MDPs — where each MDP has its own randomly sampled observation function while the policies try to minimize the KL between them. This would lead to the learning of robust representation functions. The randomly perturbed observation function is a key aspect of the paper — in their paper they apply a CNN with random weights to the observation to map the true observation to a perturbed one. The paper tests this hypothesis via PPO and shows that Deep Mutual Learning is helpful for generalization on the Procgen Benchmark.

weaknesses and questions: ## Pros

1. Tackles an important problem about having a robust perception function for RL.
2. A positive thing is that the whole ... | title: MetaTool: Facilitating Large Language Models to Master Tools with Meta-task Augmentation

summary: This paper proposes to achieve generalizable tool learning by additionally training models on meta-reasoning QA tasks. The meta-reasoning data are constructed by asking questions about the tool-using process in multiple directions, including action effect, decision-making, reversion, action input boundary, etc. Experiment results show improved tool learning performance on tasks including SAW, BW, LOG, Toolbench and BFCL.

weaknesses and questions: 1. In lines 224-226, "In order to maintain the general ability of the model in the first stage, only the parameters of the query and value projection layers of the Transformer are updated instead of full-parameter training." This constraint might also affect learning ability and make comparisons unfair. Results ensuring similar settings will make results more convincing.

2. The "LLaMA3-solution" baselines are updated fewer times (10k*3) ... | title: Meta-learning Representations for Learning from Multiple Annotators

summary: This paper proposes a meta-learning approach that utilizes noisy labels from multiple annotators to build a classifier without relying on true labels. The authors employ a probabilistic framework where latent class representations in a Mixture of Gaussians model are optimized via EM. This approach maximizes the likelihood of observed noisy labels given the latent variables, assuming these noisy annotations can guide the learning of true underlying classes.

weaknesses and questions: 1. The paper assumes isotropic variance in the latent space, simplifying computation but potentially limiting flexibility. Real-world data often exhibit complex, class-specific structures that may not align with uniform variance assumptions, particularly in nuanced classification tasks.
2. Modeling A as a K*K*r matrix may lead to over-parameterization, especially with limited data. Without visualization of learned matrices,... | title: Meta ControlNet: Enhancing Task Adaptation via Meta Learning

summary: This paper introduces Meta ControlNet, leveraging meta-learning and a novel layer-freezing approach to significantly reduce the training steps needed for ControlNet from 5000 to 1000. Additionally, it enables zero-shot control in edge tasks and rapid adaptation in complex tasks like Human Pose with only 100 finetuning steps.

weaknesses and questions: There are three critical issues for this paper:
1. No quantitative results are provided. The paper only shows qualitative results, which makes it hard to evaluate the performance of the proposed method.
2. Zero-shot capability. This paper claims that the proposed method can achieve zero-shot control in edge tasks. The model is trained on HED, Segmentation, and Depth map which belong to the edge tasks. It raises a question about whether the model has achieved zero-shot control in edge tasks. If we train ControlNet on three tasks, and then test the model on Canny ... | | meta-learning scalability benchmarks | title: Extending Contextual Self-Modulation: Meta-Learning Across Modalities, Task Dimensionalities, and Data Regimes

summary: This paper proposes to extend Contextual Self-Modulation (CSM), which is an uncertainty-handling mechanism of Neural Context Flow (NCF). NCF is a robust meta-learning framework of neural ODE that includes self-modulation with high-order Taylor expansion around contexts. The authors focus on generalizing CSM into high data regimes and designed StochasticNCF and FlashCAVIA, which take CSM in various directions. From comparative experiments, the author verified that the NCF can be successfully extended to various optimization and meta-learning techniques.

weaknesses and questions: 1. I believe this paper is not clearly written.
* iCSM: It is briefly mentioned that the space of iCSM is "an infinite-dimensional variation" that utilizes "a space of multi-layer perceptrons, whose weights are flattened into a 1-dimensional tensor." I do not find a good enough ex... | title: A solvable model of inference-time scaling

summary: This paper introduces an analytically tractable model of inference-time scaling using Bayesian linear regression with reward-weighted sampling, deriving closed-form expressions for generalization error in the high-dimensional limit. The authors prove that when the reward model is well-aligned with the teacher, error decreases monotonically with inference samples $k$ (scaling as $\Theta(1/k^2)$ in the best-of-k limit), but substantial reward misspecification induces a finite optimal $k$ and optimal temperature. The theory delineates parameter regimes where scaling inference-time compute is provably more effective than collecting additional training data, though this advantage degrades as task difficulty increases.

weaknesses and questions: 1. Oversimplified model: The paper only studies linear regression with quadratic rewards and Gaussian assumptions, while real LLMs involve highly nonlinear neural networks, complex reward mo... | title: MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?

summary: This paper presents a new evaluation benchmark for Multimodal Large Language Models (MLLMs), dubbed MME-RealWorld, which focuses on challenges that models face in the real world. Specifically, MME-RealWorld covers 29,429 question-answer pairs across 5 real-world scenarios. Experimental results on MME-RealWorld show that even the most advanced models still struggled in real-life scenarios. Besides, the authors have also conducted detailed analyses to explain the unsatisfying performance of MLLMs.

weaknesses and questions: - The evaluation on MME-RealWorld seems to require lots of computation resources, which may limit the accessibility for researchers with fewer resources.
Do the authors have plans to expand or adapt MME-RealWorld to include new tasks or modalities as MLLMs capabilities evolve? | title: On the Embedding Collapse When Scaling up Recommendation Models

summary: This paper studies recommendation model performance when scaling up the embedding layers of the model. The paper identifies a phenomenon of embedding collapse, wherein the embedding matrix tends to reside in a low-dimensional subspace. Through empirical experiments on FFM and DCNv2 and theoretical analysis on FM, the paper shows that the feature interaction process of recommendation models leads to embedding collapse and thus limits the model scalability. The paper also performed empirical experiments on regularized DCNv2 and DNN which led to less collapsed embeddings, but the model performance got worse. The paper proposes multi-embedding, which leads to better performance when scaling up the embedding layers. Experiments demonstrate that this proposed design provides consistent scalability for various recommendation models.

weaknesses and questions: - In section 3, the paper proposes Information Abundan... | title: Modality-Agnostic Self-Supervised Learning with Meta-Learned Masked Auto-Encoder

summary: This paper presents Meta-learned Masked Auto-Encoder (MetaMAE), a novel modality-agnostic self-supervised learning (SSL) framework that leverages meta-learning to improve the transfer abilities of Masked Auto-Encoder (MAE). The authors reinterpret the mask reconstruction task of MAE as a meta-learning task and propose the integration of two advanced meta-learning techniques: gradient-based meta-learning and task contrastive learning. MetaMAE is evaluated on various data modalities from modality-agnostic SSL benchmarks, demonstrating significant improvements over previous modality-agnostic SSL methods in linear evaluation. The proposed approach also shows improved transferability on cross-domain datasets.

weaknesses and questions: 1. There are significant differences in pretraining and fine-tuning hyperparameters for various downstream tasks, such as masking ratio, batch size, and decoder ... | title: GraphBench: Next-generation graph learning benchmarking

summary: This paper introduces GraphBench, a contribution of around 20 unique datasets from 7 broad and diverse categories for graph learning benchmarking. It complements the existing popular graph learning benchmarks which may be significant for molecular and citation networks, as examples, but often missing for other areas such as chip design, circuit design and weather forecasting, among others (though there are individual areas in the literature that tackle these problems). The paper also highlights the current limitations with graph benchmarks in terms of data diversity reflecting multiple real world scenarios, in/out distribution splits, evaluation consistencies and framework for usage. It finally presents a framework based on Pytorch and Pytorch Geometric which acts as the interface for loaders, optimizers and evaluators.

weaknesses and questions: - The manuscript includes reasonable discussion points on limitation... | title: (Out-of-context) Meta-learning in Language Models

summary: The paper shows the existence of a phenomenon that the authors refer to as out-of-contect meta learning in large language models. The authors design experiments that show that this phenomenon causes the internalization of text that is broadly useful, meaning that the LLM is more likely to treat this content as true. The paper shows two forms of internalization, namely weak and strong internalization, the later being a form of meta learning. Two reasons are suggested for this phenomenon, one based on the parameters of the model, and another one relying on the implicit gradient alignment bias of gradient-based optimization methods.

weaknesses and questions: * There is no conclusive explanation of the reasons why internalization happens
* The phenomenon is hard to formalize and study, which limits the advantage of the insights in the paper
None | title: Training-Free Generalization on Heterogeneous Tabular Data via Meta-Representation

summary: This paper introduces a novel approach to enable training-free generalization for tabular datasets.

The core idea is something like:

For any given dataset, the input label data (x, y) is restructured into a new format: (distance to prototypes of class c, likelihood of the label of class c). This uniform data representation allows different datasets to be organized in a consistent manner. Thus, a model trained on this standardized format can effectively generalize across various tabular datasets.

On unseen datasets, the proposed model achieves superior performances and saves training time.

weaknesses and questions: The dataset used in this study is somewhat limited. Although I have confidence in the model's ability to generalize effectively to new datasets by representing data points in terms of their similarity to prototypes, there are concerns about its adaptability to other dataset... | | anchor quality upper bounds training | title: Tournament Style RL: Stabilizing Policy Optimization on Non Verifiable Problems

summary: The paper proposes a novel tournament reward calculated against a given set of anchor answers to generate reward supervision for LLM training in tasks without verifiable rewards. For each input prompt, a set of anchors is generated before training using a stronger LLM and ranked. Then the generated answers are compared against this ranked set of anchors to generate a reward for each answer which is then used for GRPO fine-tuning.

weaknesses and questions: **Strength**
1. the proposed method is well motivated and clearly presented
2. can be easily implemented upon GRPO style fine-tuning pipelines
3. is robust against noise in evaluator LLMs

**Weakness**
1. The proposed method relies heavily on anchor model quality. And the score itself will saturate if the model being fine-tuned surpasses the anchor model's quality. On the other hand, the performance of anchor model upper limits the model ... | title: Unified Stability Bounds for Structured World Models: Geometry, Equivariance, and Identifiability as Sufficient Conditions

summary: Overview：
This paper addresses a key challenge in model-based reinforcement learning: the lack of a principled and low-overhead framework for diagnosing the quality of learned world-model representations. Motivated by the need to move beyond expensive, end-to-end evaluations and the limitations of existing theories, the authors aim to explain which properties of a representation govern downstream control performance and how to test them on existing model checkpoints. To solve this, the paper introduces a unified stability bound that decomposes the policy's suboptimality gap into three verifiable channels: geometric distortion (Îş), an identifiability defect proxied by Total Correlation (TC), and an equivariance defect proxied by Local Equivariance Error (LEE). The authors then propose a practical diagnostic protocol where these proxies are measured ... | title: Quality Control at Your Fingertips: Quality-Aware Translation Models

summary: The paper proposed two methods to make the NMT model quality aware. One is to prompt the NMT model with a quality score during training, but using the best score during inference time. The other is similar to multi-task learning but in a more unified way by appending the quality score in the target side. Both approaches show promising improvements in translation quality and one of them can work well with the MBR decoding to boost the translation quality further.

weaknesses and questions: My concerns are in the questions. If they can be addressed properly, they won't be weakness to me.
In conclusion, which one between QA prompting and prediction approaches is your recommendation in the situations including latency sensitive inference and large scale distillation. Please also describe how do you scale your methods in large scale multilingual machine translation system. The experiments highly relies on ... | title: Utility Boundary of Dataset Distillation: Scaling and Configuration-Coverage Laws

summary: This paper proposes a unified configuration–dynamics–error framework that integrates gradient, distribution, and trajectory matching within a generalization-error analysis. It establishes the scaling law and coverage law linking distilled sample size to performance and configuration diversity, theoretically and empirically unifying major dataset distillation methods.

weaknesses and questions: 1. The framework relies on PL conditions and Lipschitz continuity. While these assumptions are standard in convergence analysis, they may not strictly hold for modern deep networks with non-smooth activations, normalization layers, and stochastic training components. The practical relevance of the theoretical results could be further clarified by discussing their validity under relaxed or empirically realistic assumptions.
2. The validation of the proposed laws relies mainly on curve-fitting without... | title: Task-Robust Pre-Training for Worst-Case Downstream Adaptation

summary: In order to improve the robustness of the pre-trained model on downstream tasks, the authors propose a simple optimization algorithm, softmax weighted gradient descent, to minimize the worst-case expected risk of upstream tasks.

weaknesses and questions: In the experimental part, the method does not seem to show a consistent improvement. For example, as shown in Table 1, although the author mentioned that the model has significantly improved performance on many more challenging tasks, it has worse performance than the previous model on some downstream tasks that have performed well. See Table 2 for the same reason. Could this be improved with some tweaks for consistency?
According to the weekness I mentioned above, my question is whether such a strategy is a trade-off in the performance between the best case and the worst case on the downstream task, and cannot achieve consistency improvement? | title: Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation

summary: This paper studies activation steering for jailbreak defense under unseen or out-of-distribution attacks. Instead of learning steering only from a fixed supervised jailbreak set, the paper proposes to simulate diverse jailbroken activations through unsupervised latent direction discovery, and then train a potential-induced steering field with a bi-level adversarial objective. The method is evaluated on three aligned LLMs and six jailbreak families, and the paper reports improved robustness together with increasing subspace coverage during training.

weaknesses and questions: **Strengths**
- The paper targets an important problem in LLM safety, namely whether activation steering can generalize beyond a fixed supervised jailbreak set.
- The overall method is reasonably well motivated. In particular, combining unsupervised jailbreak activation simulation with a learned steer... | title: Learning to Quantize for Training Vector-Quantized Networks

summary: This paper proposes a novel vector quantization training framework Meta-Quantization inspired by meta-learning, which decouples the optimization of codebook and autoencoder into two stages, enabling dynamic codebook generation and task-specific training. The proposed method outperforms existing vector quantization approaches on image construction and generation tasks.

weaknesses and questions: The description of convergence is inconsistent. While Figure 2 states that $\phi$ and $\theta$ are trained to convergence before training $\psi$, Algorithm 1 shows that they are updated together. This discrepancy creates ambiguity regarding the actual optimization procedure implemented in the paper.
In the introduction, it is mentioned that the codebook utilization in previous methods is low. However, in the experiments (Table 3, 4, 5), the codebook utilization of VQGAN-LC is also quite high. Please provide a justificat... | title: Estimating Fréchet bounds for validating programmatic weak supervision

summary: This paper proposes solutions via convex programs to estimate Frechet bounds for Programmatic Weak Supervision (PWS). This approach uses estimates of the true labels via labelmodels (i.e., different aggregation schemes that exist in the literature). With these estimates of the labels, they provide an approach to estimate bounds on the accuracy (and other quantities) of the weak labelers. They provide experiments to check the validity of their bounds and also provide experiments with weak labelers generated via prompting to examine how their bounds perform under instances of weak labelers with different qualities/accuracies.

weaknesses and questions: 1. One weakness is that this approach is fundamentally reliant on the quality of the label model. This is manifested in assumption 2.3, which states that the estimate of the conditional distribution of $Y \| Z$ should approach the true conditional distri... | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "GuideGISTEmbedLoss", "matryoshka_dims": [ 768, 512, 256, 128 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 1024 - `num_train_epochs`: 1.0 - `learning_rate`: 2e-05 - `lr_scheduler_type`: cosine - `warmup_steps`: 0.1 - `bf16`: True - `eval_on_start`: True - `dataloader_num_workers`: 4 - `ddp_find_unused_parameters`: False - `prompts`: {'anchor': 'task: search result | query: ', 'positive': 'title: none | text: ', 'negative_1': 'title: none | text: ', 'negative_2': 'title: none | text: ', 'negative_3': 'title: none | text: ', 'negative_4': 'title: none | text: ', 'negative_5': 'title: none | text: ', 'negative_6': 'title: none | text: ', 'negative_7': 'title: none | text: '} - `batch_sampler`: no_duplicates_hashed #### All Hyperparameters
Click to expand - `per_device_train_batch_size`: 1024 - `num_train_epochs`: 1.0 - `max_steps`: -1 - `learning_rate`: 2e-05 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: None - `warmup_steps`: 0.1 - `optim`: adamw_torch_fused - `optim_args`: None - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `optim_target_modules`: None - `gradient_accumulation_steps`: 1 - `average_tokens_across_devices`: True - `max_grad_norm`: 1.0 - `label_smoothing_factor`: 0.0 - `bf16`: True - `fp16`: False - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `use_liger_kernel`: False - `liger_kernel_config`: None - `use_cache`: False - `neftune_noise_alpha`: None - `torch_empty_cache_steps`: None - `auto_find_batch_size`: False - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `include_num_input_tokens_seen`: no - `log_level`: passive - `log_level_replica`: warning - `disable_tqdm`: False - `project`: huggingface - `trackio_space_id`: None - `trackio_bucket_id`: None - `trackio_static_space_id`: None - `per_device_eval_batch_size`: 8 - `prediction_loss_only`: True - `eval_on_start`: True - `eval_do_concat_batches`: True - `eval_use_gather_object`: False - `eval_accumulation_steps`: None - `include_for_metrics`: [] - `batch_eval_metrics`: False - `save_only_model`: False - `save_on_each_node`: False - `enable_jit_checkpoint`: False - `push_to_hub`: False - `hub_private_repo`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_always_push`: False - `hub_revision`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `restore_callback_states_from_checkpoint`: False - `full_determinism`: False - `seed`: 42 - `data_seed`: None - `use_cpu`: False - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `dataloader_drop_last`: True - `dataloader_num_workers`: 4 - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `dataloader_prefetch_factor`: None - `remove_unused_columns`: True - `label_names`: None - `train_sampling_strategy`: random - `length_column_name`: length - `ddp_find_unused_parameters`: False - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `ddp_static_graph`: None - `ddp_backend`: None - `ddp_timeout`: 1800 - `fsdp`: None - `fsdp_config`: None - `deepspeed`: None - `debug`: [] - `skip_memory_metrics`: True - `do_predict`: False - `resume_from_checkpoint`: None - `warmup_ratio`: None - `local_rank`: -1 - `prompts`: {'anchor': 'task: search result | query: ', 'positive': 'title: none | text: ', 'negative_1': 'title: none | text: ', 'negative_2': 'title: none | text: ', 'negative_3': 'title: none | text: ', 'negative_4': 'title: none | text: ', 'negative_5': 'title: none | text: ', 'negative_6': 'title: none | text: ', 'negative_7': 'title: none | text: '} - `batch_sampler`: no_duplicates_hashed - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {}
### Training Logs | Epoch | Step | Training Loss | reviewsearch_cosine_ndcg@10 | |:------:|:----:|:-------------:|:---------------------------:| | 0 | 0 | - | 0.1704 | | 0.0909 | 2 | 34.5974 | - | | 0.1364 | 3 | - | 0.2356 | | 0.1818 | 4 | 25.3299 | - | | 0.2727 | 6 | 16.6895 | 0.2372 | | 0.3636 | 8 | 13.9725 | - | | 0.4091 | 9 | - | 0.2423 | | 0.4545 | 10 | 12.2704 | - | | 0.5455 | 12 | 11.2867 | 0.2473 | | 0.6364 | 14 | 10.7076 | - | | 0.6818 | 15 | - | 0.2492 | | 0.7273 | 16 | 10.2320 | - | | 0.8182 | 18 | 10.0333 | 0.2509 | | 0.9091 | 20 | 9.9865 | - | | 0.9545 | 21 | - | 0.2507 | | 1.0 | 22 | 9.9844 | 0.2512 | ### Training Time - **Training**: 4.2 hours - **Evaluation**: 1.6 hours - **Total**: 5.7 hours ### Framework Versions - Python: 3.12.9 - Sentence Transformers: 5.6.0 - Transformers: 5.12.1 - PyTorch: 2.8.0+cu128 - Accelerate: 1.14.0 - Datasets: 5.0.0 - Tokenizers: 0.22.2 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ```