Journal articles on the topic 'Rademacher averages'

We develop a technique for deriving data-dependent error bounds for transductive learning algorithms based on transductive Rademacher complexity. Our technique is based on a novel general error bound for transduction in terms of transductive Rademacher complexity, together with a novel bounding technique for Rademacher averages for particular algorithms, in terms of their "unlabeled-labeled" representation. This technique is relevant to many advanced graph-based transductive algorithms and we demonstrate its effectiveness by deriving error bounds to three well known algorithms. Finally, we present a new PAC-Bayesian bound for mixtures of transductive algorithms based on our Rademacher bounds.

In this paper, a dispersed system formed by an ensemble of particles of different volume has been modeled in the framework of a thermodynamical approach. Particle shape has been determined by its fractal dimension which correlates its volume and surface area. Using the methods of number theory and Hardy-Ramanujan-Rademacher formula, we have calculated the equilibrium size distributions for nanoparticles of different shape in an ensemble. Estimates of the average volume and fractal dimension of dispersed particles have been obtained based on distribution functions. The correlation between average geometrical characteristics of particles in the ensemble, thermodynamical conditions of the dispersed system and properties of its substance have also been revealed.

In this letter, we consider a density-level detection (DLD) problem by a coefficient-based classification framework with [Formula: see text]-regularizer and data-dependent hypothesis spaces. Although the data-dependent characteristic of the algorithm provides flexibility and adaptivity for DLD, it leads to difficulty in generalization error analysis. To overcome this difficulty, an error decomposition is introduced from an established classification framework. On the basis of this decomposition, the estimate of the learning rate is obtained by using Rademacher average and stepping-stone techniques. In particular, the estimate is independent of the capacity assumption used in the previous literature.

Recently, learning to rank, which aims at constructing a model for ranking objects, is one of the hot research topics in information retrieval and machine learning communities. Most of existing learning to rank approaches are based on the assumption that each object is independently and identically distributed. Although this assumption simplifies ranking problems, the implicit interconnections between objects are ignored. In this paper, a graph based ranking framework is proposed, which takes advantage of implicit correlations between objects. Furthermore, the derived relational ranking algorithm from this framework, called GRSVM, is developed based on the conventional algorithm RankSVM-primal. In addition, generalization properties of different relational ranking algorithms are analyzed using Rademacher Average. Based on the analysis, we find that GRSVM can achieve tighter generalization bound than existing relational ranking algorithms in most cases. Finally, a comparison of experimental results produced by improved and conventional algorithms shows the superior performance of the former.

Positive semi-definite matrices commonly occur as normal matrices of least squares problems in statistics or as kernel matrices in machine learning and approximation theory. They are typically large and dense. Thus algorithms to solve systems with such a matrix can be very costly. A core idea to reduce computational complexity is to approximate the matrix by one with a low rank. The optimal and well understood choice is based on the eigenvalue decomposition of the matrix. Unfortunately, this is computationally very expensive. Cheaper methods are based on Gaussian elimination but they require pivoting. We show how invariant matrix theory provides explicit error formulas for an averaged error based on volume sampling. The formula leads to ratios of elementary symmetric polynomials on the eigenvalues. We discuss several bounds for the expected norm of the approximation error and include examples where this expected error norm can be computed exactly. References A. Dax. “On extremum properties of orthogonal quotients matrices”. In: Lin. Alg. Appl. 432.5 (2010), pp. 1234–1257. doi: 10.1016/j.laa.2009.10.034. M. Dereziński and M. W. Mahoney. Determinantal Point Processes in Randomized Numerical Linear Algebra. 2020. url: https://arxiv.org/abs/2005.03185. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang. “Matrix approximation and projective clustering via volume sampling”. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm. SODA ’06. Miami, Florida: Society for Industrial and Applied Mathematics, 2006, pp. 1117–1126. url: https://dl.acm.org/doi/abs/10.5555/1109557.1109681. S. A. Goreinov, E. E. Tyrtyshnikov, and N. L. Zamarashkin. “A theory of pseudoskeleton approximations”. In: Lin. Alg. Appl. 261.1 (1997), pp. 1–21. doi: 10.1016/S0024-3795(96)00301-1. M. W. Mahoney and P. Drineas. “CUR matrix decompositions for improved data analysis”. In: Proc. Nat. Acad. Sci. 106.3 (Jan. 20, 2009), pp. 697–702. doi: 10.1073/pnas.0803205106. M. Marcus and L. Lopes. “Inequalities for symmetric functions and Hermitian matrices”. In: Can. J. Math. 9 (1957), pp. 305–312. doi: 10.4153/CJM-1957-037-9.

Background:Observational cohort studies have shown that there is low, but still detectable progression level in radiographic sacroiliitis, which might also have an impact on the function in patients with axial spondyloarthritis (axSpA). Recent data showed that tumor necrosis factor inhibitors (TNFi) might retard spinal progression when initiated earlier and taken longer in patients with axSpA. However, the question of whether they also have such an effect on radiographic progression in sacroiliac joints (SIJs) is still unclear.Objectives:To investigate the longitudinal association between radiographic sacroiliitis progression and treatment with TNFi in patients with early axial SpA in a long-term inception cohort.Methods:Based on the availability of at least two sets of SIJ radiographs, 301 patients (166 with nr-axSpA, symptom duration ≤5 years and 135 with r-axSpA, symptom duration ≤10 years) from the German Spondyloarthritis Inception Cohort (GESPIC) were included in this analysis. These patients contributed with a total of 737 2-year radiographic intervals. Two trained and calibrated central readers scored the radiographs according to the modified New York criteria. If both scored an image as definite radiographic sacroiliitis, the patient was classified as having r-axSpA. The sacroiliac sum score was calculated as a mean of both readers. The association between previous as well as current TNFi use and radiographic sacroiliitis progression, which was defined as the change in the sacroiliitis sum score over 2 years, was analysed using longitudinal generalized estimating equations (GEE) analysis.Results:At baseline, 9 (3.0%) patients were treated with a TNFi, and 87 (28.9%) patients received at least one TNFi during the entire follow-up period. A total of 141 of the radiographic intervals were covered with TNFi of any duration, while 109 of them were covered with a TNFi of at least 12 months. While receiving ≥12 months TNFi in the previous interval was associated with a lower progression of the sacroiliitis sum score compared to not receiving TNFi in the previous interval, this was not the case in patients who received TNFi ≥12 months in the current 2-year interval (Figure 1). The significant association between TNF ≥12 months in the previous interval and progression in the sacroiliitis sum score were confirmed in the adjusted multivariable longitudinal GEE analysis. In addition, a similar trend for the beneficial effects was observed in different models, which included other treatment definitions with TNFi in the previous 2-year interval (Table).Table 1.The longitudinal GEE analysis of the association between progression in the sacroiliitis sum score and TNFi use.TNFi treatment definitionReferenceβ* (95% CI)TNFi for ≥ 12 months in the previous 2-year intervalNo TNFi for ≥ 12 months in the previous 2-year interval-0.09 (-0.18, -0.003)Any TNFi use in the previous 2-year intervalNo TNFi use in the previous 2-year interval-0.09 (-0.17, 0.002)TNFi for ≥ 12 months in the current 2-year intervalNo TNFi for ≥ 12 months in the current 2-year interval-0.03 (-0.11, 0.06)Any TNFi use in the current 2-year intervalNo TNFi use in the current 2-year interval0.05 (-0.05, 0.14)TNFi for ≥ 12 months in the previous and ≥ 12 months in the current 2-year intervalNo TNFi for ≥ 12 months in the previous and ≥ 12 months in the current 2-year interval-0.08 (-0.17, 0.004)* Parameter estimates from the multivariable models adjusted for sex, age at the beginning of the current 2-year interval, HLA-B27 positivity, symptom duration at the beginning of the current 2-year interval, time-averaged elevated CRP, time-averaged BASDAI, and time-averaged NSAID intake score in the current 2-year interval.Conclusion:Treatment with TNFi was associated with retardation of radiographic sacroiliitis progression in patients with axSpA. This effect becomes evident between 2 and 4 years after treatment initiation.References:Acknowledgements:GESPIC was initially supported by the BMBF. As a consequence of the funding reduction by BMBF according to schedule in 2005 and stopped in 2007, complementary financial support has been obtained also from Abbott, Amgen, Centocor, Schering–Plough, and Wyeth. Starting from 2010, the core GESPIC cohort was supported by AbbVie.Disclosure of Interests:Murat Torgutalp: None declared, Valeria Rios Rodriguez: None declared, Maryna Verba: None declared, Mikhail Protopopov: None declared, Fabian Proft: None declared, Judith Rademacher: None declared, Hildrun Haibel: None declared, Martin Rudwaleit Consultant of: AbbVie, BMS, Celgene, Janssen, Eli Lilly, MSD, Novartis, Pfizer, Roche, UCB Pharma, Joachim Sieper: None declared, Denis Poddubnyy Speakers bureau: AbbVie, Bristol-Myers Squibb, Lilly, MSD, Novartis, Pfizer, and UCB, Consultant of: AbbVie, Biocad, Gilead, GlaxoSmithKline, Eli Lilly, MSD, Novartis, Pfizer, Samsung Bioepis, and UCB, Grant/research support from: AbbVie, MSD, Novartis, and Pfizer

Abstract Background: SCD is caused by a point mutation in the β-globin gene producing hemoglobin S (HbS) that polymerizes upon deoxygenation with subsequent formation of sickled red blood cells (RBCs). GBT440 is a novel, orally bioavailable small molecule that inhibits HbS polymerization by increasing the affinity of O2 to hemoglobin (Hb). Methods: The pharmacokinetics, mass balance, and metabolite profile of [14C]-GBT440 were evaluated in 7 healthy male subjects in this open-label study. In order to evaluate the disposition kinetics of GBT440 at steady-state concentrations, a loading/maintenance dose schema was employed. Each subject received an oral loading dose of 2000 mg GBT440 on Day 1 followed by oral maintenance doses of 400 mg once daily on Day 2 to Day 4. Once the target steady-state was achieved, a single [14C]-GBT440 400 mg dose (approximately 100 μCi) was administered orally on Day 5. Blood, plasma, urine and feces were collected serially up to 26 days postdose. Results: There were no serious adverse events or discontinuations due to adverse events for any of the healthy subjects participating in this study. GBT440 reached Cmax in plasma and whole blood with median time to maximum concentration (Tmax) values of 2.00 hours in plasma and whole blood and in 6.00 hours in RBCs. After reaching Cmax, GBT440 concentrations appeared to decline in a monophasic manner, with the terminal elimination phase for GBT440 in plasma, whole blood, and RBCs appearing to decline in a parallel manner, with geometric mean T1/2 values of 98.0 hours in plasma, 66.3 hours in whole blood, and 65.8 hours in RBCs. This study achieved 98.0% average recovery of total radioactivity in urine and feces over the course of the study. Most of the administered radioactivity (88.2%) was recovered by 144 hours postdose (Day 7). GBT440 was eliminated primarily in feces (62.6% of the total radioactive dose) with urinary excretion accounting for 35.4% of the total radioactive dose. In whole blood, the majority of the total radioactivity (TRA) was unchanged GBT440 (97.5%) while three metabolites accounted for the remaining TRA (2.5%). In plasma, unchanged GBT440 was the prominent circulating radioactive component, accounting for 48.8% of the TRA. Eleven circulating metabolites with corresponding radioactive peaks were identified. There was one major Phase II metabolite (GBT440 O-dealkylation-sulfation), accounting for 16.8% of the TRA. Two potential active metabolites were identified but only accounted for 2.5% of the dose in whole blood. GBT440 was eliminated predominately in feces. Unchanged GBT440 was the most abundant radioactive component, accounting for 33.3% of the administered dose. Four metabolites were identified, each accounting for 5.62%, 2.66%, 1.66% and less than 6% of the dose in the 0-216-hr human feces. Urine was a relatively minor excretion route for GBT440 in humans. An average of 34.3% of the dose was recovered in the urine samples. Unchanged GBT440 accounted for 0.08% of the administered dose and the rest were metabolites. GBT440 glucuronidation and reduction-glucuronidation products, which are Phase II metabolites, were the most abundant metabolites in urine, accounting for a combined 9.22% of dose. Because GBT440 does not undergo renal elimination, patients with renal disorders should not experience changes in pharmacokinetics of GBT440. Conclusions: Although GBT440 has high specific binding to hemoglobin, it was completely excreted from the body with a half-life of approximately three days in healthy subjects. Since the half-life of GBT440 was much shorter than RBC lifespan (~ 120 days), this supports the hypothesis that the binding between GBT440 to hemoglobin is a reversible process. Following an oral administration, approximately one-third of the dose was excreted as the unchanged drug into the feces (unabsorbed and/or via biliary excretion). Two-thirds of the administered dose was metabolized and excreted into urine and feces. The major metabolic pathway was via Phase I and Phase II metabolism. Because GBT440 was not excreted directly into the urine, the pharmacokinetics are unlikely to be affected in patients with renal disorders. Disclosures Rademacher: Global Blood Therapeutics: Employment, Equity Ownership. Hutchaleelaha:Global Blood Therapeutics: Employment, Equity Ownership. Washington:Global Blood Therapeutics: Employment, Equity Ownership. Lehrer:Global Blood Therapeutics: Employment, Equity Ownership. Ramos:Global Blood Therapeutics: Employment, Equity Ownership.

Background:There are inconclusive data on the effect of tumor necrosis factor inhibitors (TNFi) on radiographic spinal progression in axial spondyloarthritis (axSpA). Although inflammation and new bone formation are linked in axSpA, TNFi failed to show inhibition of radiographic spinal progression over two years compared to historical cohorts in pivotal studies in radiographic axSpA. Subsequent observational studies suggested that a longer treatment duration, earlier treatment initiation and effective inflammation suppression might be required to achieve inhibition of radiographic progression.Objectives:The aim of the current study was to evaluate the effect of TNFi on radiographic spinal progression in patients with early axSpA in a long-term inception cohort.Methods:A total of 266 patients with early axSpA (with r-axSpA with symptom duration ≤10 years and nr-axSpA with symptom duration ≤5 years) from the German Spondyloarthritis Inception Cohort (GESPIC) with at least two sets of spinal radiographs obtained at least 2 years apart during a 10-year follow-up were included. These patients contributed with a total of 542 2-year radiographic intervals. Spinal radiographs were evaluated by three trained and calibrated readers according to the modified Stoke Ankylosing Spondylitis Spine Score (mSASSS). The final mSASSS was calculated as a mean of three reader scores. The association between the current TNFi, previous TNFi and radiographic spinal progression defined as the absolute mSASSS change score over 2 years was analyzed using longitudinal generalized estimating equations (GEE) analysis.Results:Only 9 patients were treated with a tumor necrosis factor inhibitor (TNFi) at baseline, and a total of 77 patients received TNFi during the entire follow-up period that gave 103 2-year intervals covered by TNFi of any duration, and 78 intervals covered by TNFi with treatment duration of at least 12 months. Radiographic spinal progression in axSpA patients receiving TNFi in the current 2-year interval was not different from progression in patients not treated with TNFi, while TNFi in the previous 2-year interval was associated with lower progression compared to patients without TNFi in this interval (Figure 1). The latter was also evident for patients who received TNFi in both previous and current 2-year intervals, i.e. patients treated with TNFi over 4 years. The longitudinal GEE analysis confirmed no significant association between current TNFi treatment and radiographic spinal progression but a significant association between TNFi in the previous 2-year interval (especially if this was continued also in the current interval giving 4 years in total) and the progression in the current one (Table 1).Table 1.The association between the change of the mSASSS over two years and current and/or previous treatment with TNFi in the longitudinal generalized estimation equation analysis.TNFi treatment definitionReferenceβ*(95% CI)TNFi for ≥12 months in the previous 2-year intervalTNFi for ≥12 months in the current 2-year intervalYesNo TNFi for ≥12 months in the current 2-year interval-0.19(-0.56 to 0.18)YesNo TNFi for ≥12 months in the previous 2-year interval-0.56(-0.95 to -0.17)YesYesNo TNFi for ≥12 months in the current and previous 2-year intervals-0.59(-1.03 to -0.15)*Parameter estimates from the multivariable models adjusted for sex, symptom duration at the beginning of the current 2-year interval, time-averaged ASDAS in the current 2-year interval, smoking in the current 2-year interval, classification as non-radiographic or radiographic axSpA, and mSASSS at the beginning of the current 2-year interval.Conclusion:TNFi treatment exhibits a time-shifted inhibitory effect on radiographic spinal progression in axSpA that becomes evident only in the second 2-year interval after treatment initiation.Acknowledgements:GESPIC was initially supported by the BMBF. As consequence of the funding reduction by BMBF according to schedule in 2005 and stopped in 2007, complementary financial support has been obtained also from Abbott, Amgen, Centocor, Schering–Plough, and Wyeth. Starting from 2010, the core GESPIC cohort was supported by AbbVie.Disclosure of Interests:Denis Poddubnyy Speakers bureau: AbbVie, Bristol-Myers Squibb, Lilly, MSD, Novartis, Pfizer, and UCB, Consultant of: AbbVie, Biocad, Gilead, GlaxoSmithKline, Eli Lilly, MSD, Novartis, Pfizer, Samsung Bioepis, and UCB, Grant/research support from: AbbVie, MSD, Novartis, and Pfizer, Valeria Rios Rodriguez: None declared, Murat Torgutalp: None declared, Ani Dilbaryan: None declared, Maryna Verba: None declared, Fabian Proft: None declared, Mikhail Protopopov: None declared, Judith Rademacher: None declared, Hildrun Haibel: None declared, Joachim Sieper: None declared, Martin Rudwaleit Consultant of: AbbVie, BMS, Celgene, Janssen, Eli Lilly, MSD, Novartis, Pfizer, Roche, UCB Pharma

BackgroundThere are conflicting data regarding effect of nonsteroidal anti-inflammatory drugs (NSAID) on radiographic spinal progression in axial spondyloarthritis (axSpA). The analysis of the first 2-year of the GErman SPondyloarthritis Inception Cohort (GESPIC) showed that higher NSAID intake may retard new bone formation in r-axSpA. It remained, however, unclear, whether cyclooxygenase-2 selective inhibitors (COX2i) might have a stronger effect than non-selective (NS) ones and if the effect could be observed also in nr-axSpA.ObjectivesTo investigate the effect of NSAIDs (COX2i and NS) intake on radiographic spinal progression in patients with r-axSpA and nr-axSpA.MethodsBased on availability of at least two sets of spinal radiographs during 10-year follow-up, 243 patients with early axSpA (130 and 113 nr- and r-axSpA, respectively) from GESPIC were included in this analysis. The patients contributed a total of 540 2-year radiographic intervals. Radiographs were scored by 3 trained and calibrated readers according to modified Stoke Ankylosing Spondylitis Spine Score (mSASSS). Final mSASSS was calculated as a mean of 3 readers, and progression was defined as absolute mSASSS change score over 2 years. NSAID type, daily dose, and frequency of intake were recorded at visits. The ASAS index of NSAID intake (0-100) counting both dose and duration of intake was calculated for intervals. The association between NSAID intake (NSAID type and NSAID score) and radiographic spinal progression over 2 years was analysed using longitudinal generalized estimated equations (GEE).ResultsAt baseline, 161 (66.3%) patients were treated with NSAIDs. While 289 (53.5%) and 128 (23.7%) 2-year radiographic intervals were covered by NS and COX-2i respectively, 123 (22.8%) intervals were not covered by NSAID. The significant association between higher NSAID intake and retardation of radiographic spinal progression was found in adjusted multivariable longitudinal GEE analysis. This effect was mostly attributable to patients with r-axSpA (Table 1). mSASSS progression was numerically lower in patients taking COX2i (irrespectively of dose) as compared to patients treated with NS-NSAIDs; in stratified analysis, however, there was no clear dose-dependency (as reflected by NSAID index) in both groups (Figure 1, Table 1).Table 1.The association between radiographic spinal progression (mSASSS change score) and NSAID intake in patients with axSpA in multivariable longitudinal GEEAll axSpA β (95% CI)* (n=461)nr-axSpA β (95% CI)*(n=244)r-axSpA β (95% CI)* (n=217)NSAID intake score, per 10 points-0.04 (-0.09, 0.00)-0.02 (-0.06, 0.02)-0.07 (-0.13, 0.00)NSAID type§ NS inhibitors vs No NSAID0.30(-0.07, 0.66)0.25(-0.07, 0.57)0.26(-0.40, 0.92) COX2i vs No NSAID0.17(-0.19, 0.54)0.15(-0.15, 0.46)0.18(-0.49, 0.85) COX2i vs NS inhibitors-0.12(-0.37, 0.12)-0.10(-0.28, 0.09)-0.08(-0.57, 0.40)Analysis stratified according to NSAID typeNon-selective NSAID intake score, per 10 points-0.06(-0.12, 0.00)-0.04(-0.09, 0.01)-0.07(-0.17, 0.03)COX2 selective NSAID intake score, per 10 points-0.06(-0.13, 0.02)-0.03(-0.07, 0.02)-0.09(-0.18, 0.01)axSpA: axial spondyloarthritis; COX2i, cyclooxygenase-2 selective inhibitors; n, number of current 2-year radiographic intervals in multivariable analyses; NS, non-selective COXi; NSAID, non-steroidal anti-inflammatory drugs.*All multivariable models were adjusted for sex, symptom duration at the beginning of the interval, time-averaged ASDAS the interval, classification as radiographic axSpA, smoking in the interval, mSASSS at the beginning of theinterval, and TNFi use in the interval.§NSAID intake score was added in this model.ConclusionHigher NSAID intake is associated with lower radiographic spinal progression, particularly in r-axSpA patients. COX2i might possess a stronger inhibitory effect on radiographic progression as compared to NS-NSAIDs.Disclosure of InterestsMurat Torgutalp: None declared, Valeria Rios Rodriguez Consultant of: AbbVie, Grant/research support from: Falk e.V, Ani Dilbaryan: None declared, Fabian Proft Speakers bureau: Novartis, Lilly, UCB AbbVie, AMGEN, BMS, Hexal, MSD, Pfizer, Roche and Janssen, Grant/research support from: Novartis, Lilly and UCB, Mikhail Protopopov Consultant of: Novartis and UCB, Maryna Verba: None declared, Judith Rademacher Consultant of: Novartis and UCB, Hildrun Haibel Consultant of: Boehringer, Janssen, MSD, Novartis, Sobi, Roche, Pfizer, AbbVie, and Sobi, Joachim Sieper Speakers bureau: Abbvie, Janssen, Lilly, Merck, Novartis, UCB, Consultant of: AbbVie, Lilly, Merck, Novartis, UCB, Martin Rudwaleit Speakers bureau: AbbVie, Boehringer Ingelheim, Celgen, Chugai, Eli Lilly, Janssen, MSD, Novartis, Pfizer, UCB., Consultant of: AbbVie, Boehringer Ingelheim, Celgen, Chugai, Eli Lilly, Janssen, MSD, Novartis, Pfizer, UCB., Denis Poddubnyy Speakers bureau: AbbVie, Bristol-Myers Squibb, Eli Lilly, MSD, Novartis, Pfizer, and UCB., Consultant of: AbbVie, Biocad, Eli Lilly, Gilead, GlaxoSmithKline, Janssen, MSD, Novartis, Pfizer, Samsung Bioepis, and UCB, Grant/research support from: AbbVie, Eli Lilly, MSD, Novartis, Pfizer.

We present Bavarian , a collection of sampling-based algorithms for approximating the Betweenness Centrality (BC) of all vertices in a graph. Our algorithms use Monte-Carlo Empirical Rademacher Averages (MCERAs), a concept from statistical learning theory, to efficiently compute tight bounds on the maximum deviation of the estimates from the exact values. The MCERAs provide a sample-dependent approximation guarantee much stronger than the state of the art, thanks to its use of variance-aware probabilistic tail bounds. The flexibility of the MCERAs allows us to introduce a unifying framework that can be instantiated with existing sampling-based estimators of BC, thus allowing a fair comparison between them, decoupled from the sample-complexity results with which they were originally introduced. Additionally, we prove novel sample-complexity results showing that, for all estimators, the sample size sufficient to achieve a desired approximation guarantee depends on the vertex-diameter of the graph, an easy-to-bound characteristic quantity. We also show progressive-sampling algorithms and extensions to other centrality measures, such as percolation centrality. Our extensive experimental evaluation of Bavarian shows the improvement over the state-of-the art made possible by the MCERAs (2–4x reduction in the error bound), and it allows us to assess the different trade-offs between sample size and accuracy guarantees offered by the different estimators.

AbstractRandomized trace estimation is a popular and well-studied technique that approximates the trace of a large-scale matrix B by computing the average of $$x^T Bx$$ x T B x for many samples of a random vector X. Often, B is symmetric positive definite (SPD) but a number of applications give rise to indefinite B. Most notably, this is the case for log-determinant estimation, a task that features prominently in statistical learning, for instance in maximum likelihood estimation for Gaussian process regression. The analysis of randomized trace estimates, including tail bounds, has mostly focused on the SPD case. In this work, we derive new tail bounds for randomized trace estimates applied to indefinite B with Rademacher or Gaussian random vectors. These bounds significantly improve existing results for indefinite B, reducing the number of required samples by a factor n or even more, where n is the size of B. Even for an SPD matrix, our work improves an existing result by Roosta-Khorasani and Ascher (Found Comput Math, 15(5):1187–1212, 2015) for Rademacher vectors. This work also analyzes the combination of randomized trace estimates with the Lanczos method for approximating the trace of f(B). Particular attention is paid to the matrix logarithm, which is needed for log-determinant estimation. We improve and extend an existing result, to not only cover Rademacher but also Gaussian random vectors.

People believe that depth plays an important role in success of deep neural networks (DNN). However, this belief lacks solid theoretical justifications as far as we know. We investigate role of depth from perspective of margin bound. In margin bound, expected error is upper bounded by empirical margin error plus Rademacher Average (RA) based capacity term. First, we derive an upper bound for RA of DNN, and show that it increases with increasing depth. This indicates negative impact of depth on test performance. Second, we show that deeper networks tend to have larger representation power (measured by Betti numbers based complexity) than shallower networks in multi-class setting, and thus can lead to smaller empirical margin error. This implies positive impact of depth. The combination of these two results shows that for DNN with restricted number of hidden units, increasing depth is not always good since there is a tradeoff between positive and negative impacts. These results inspire us to seek alternative ways to achieve positive impact of depth, e.g., imposing margin-based penalty terms to cross entropy loss so as to reduce empirical margin error without increasing depth. Our experiments show that in this way, we achieve significantly better test performance.

Given [Formula: see text], we study two classes of large random matrices of the form [Formula: see text] where for every [Formula: see text], [Formula: see text] are iid copies of a random variable [Formula: see text], [Formula: see text], [Formula: see text] are two (not necessarily independent) sets of independent random vectors having different covariance matrices and generating well concentrated bilinear forms. We consider two main asymptotic regimes as [Formula: see text]: a standard one, where [Formula: see text], and a slightly modified one, where [Formula: see text] and [Formula: see text] while [Formula: see text] for some [Formula: see text]. Assuming that vectors [Formula: see text] and [Formula: see text] are normalized and isotropic “in average”, we prove the convergence in probability of the empirical spectral distributions of [Formula: see text] and [Formula: see text] to a version of the Marchenko–Pastur law and the so-called effective medium spectral distribution, correspondingly. In particular, choosing normalized Rademacher random variables as [Formula: see text], in the modified regime one can get a shifted semicircle and semicircle laws. We also apply our results to the certain classes of matrices having block structures, which were studied in [G. M. Cicuta, J. Krausser, R. Milkus and A. Zaccone, Unifying model for random matrix theory in arbitrary space dimensions, Phys. Rev. E 97(3) (2018) 032113, MR3789138; M. Pernici and G. M. Cicuta, Proof of a conjecture on the infinite dimension limit of a unifying model for random matrix theory, J. Stat. Phys. 175(2) (2019) 384–401, MR3968860].