researchmap

Publishing type：Research paper (international conference proceedings)  

The Volterra filter is a digital filter that can describe nonlinearity. In this work, we analyze the dynamic behaviors of an adaptive signal processing system with the Volterra filter and a time-varying unknown system by a statistical-mechanical method. Specifically, assuming the self-averaging property with an infinitely long tapped-delay line, we derive simultaneous dif-ferential equations that describe the behaviors of the macroscopic variables in a deterministic and closed form and obtain the exact solution by solving them analytically. In addition, the validity of the derived theory is confirmed by comparison with numerical simulations.

Scopus

researchmap

Language：Japanese   Publisher：The Society of Instrument and Control Engineers  

<p>In this paper, we assume a situation that multiple quadrotors are flying autonomously in the same space for the purpose of aerial photographing, surveillance, home delivery and so on. In such a situation, an algorithm to avoid mutual collision is needed. D. Zhou et al. proposed an algorithm based on the buffered Voronoi cells (BVC) to reach each target position without mutual collision when the multiple quadrotors fly and confirmed its operation by some simulations. In this research, we verified the performance of the method by numerical simulations using MATLAB. In addition, we confirmed the algorithm by dynamic simulations with four quadrotors using the dynamic simulator V-REP.</p>

DOI： 10.9746/sicetr.56.31

CiNii Books

researchmap

Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

DOI： 10.1109/SII46433.2020.9025842

researchmap

Publishing type：Research paper (scientific journal)  

DOI： 10.1016/j.neunet.2019.05.004

researchmap

Other Link： https://dblp.uni-trier.de/db/journals/nn/nn118.html#MotonakaM19

Language：Japanese   Publisher：The Japan Society of Mechanical Engineers  

In this paper, we assume a situation that multiple quadrotors are flying autonomously in the same space for the purpose of aerial photographing, surveillance, or home delivery and so on. In such a situation, an algorithm to avoid mutual collision is needed. D. Zhou et al. proposed an algorithm based on the Buffered Voronoi Cells (BVC) to reach each target position without mutual collision when the multiple quadrotors fly and confirmed its operation by some simulations. In this research, we applied the method based on BVC to the different number of quadrotors from the verification by D. Zhou et al. and verified the performance of it by numerical simulations using MATLAB. In addition, we confirmed the algorithm by a dynamic simulation with four quadrotors using the dynamic simulator V-REP.

DOI： 10.1299/jsmermd.2019.2a2-c05

researchmap

Publishing type：Research paper (scientific journal)   Publisher：Institute of Electronics, Information and Communications Engineers (IEICE)  

DOI： 10.1587/transfun.e101.a.2419

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：Elsevier Ltd  

Learning curves of simple perceptron were derived here. The learning curve of the perceptron learning with noisy teacher was shown to be non-monotonic, which has never appeared even though the learning curves have been analyzed for half a century. In this paper, we showed how this phenomenon occurs by analyzing the asymptotic property of the perceptron learning using a method in systems science, that is, calculating the eigenvalues of the system matrix and the corresponding eigenvectors. We also analyzed the AdaTron learning and the Hebbian learning in the same way and found that the learning curve of the AdaTron learning is non-monotonic whereas that of the Hebbian learning is monotonic.

DOI： 10.1016/j.neunet.2018.02.009

Scopus

researchmap

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)   Publisher：Institute of Electrical Engineers of Japan  

We analyze the adaptation rate of active noise control using a statistical-mechanical method. Three measures are employed to evaluate the adaptation rate. The first measure is the MSE initial decreasing rate. The second measure is an adaptation constant, which is defined as the negative of the maximum eigenvalue of the coefficient matrix of differential equations that describe the dynamical behaviors of the macroscopic variables. The third measure is the integral of the MSE, which is defined as the integral of the difference between the MSE and the steady-state MSE. The first and second measures focus on only the initial and final stages of the MSE, respectively. In contrast, the third measure considers all stages in a well-balanced manner. Therefore, employing the integral of the MSE, we can determine the step size that optimizes the entire learning curve.

DOI： 10.1541/ieejeiss.138.369

Scopus

researchmap

Publishing type：Research paper (international conference proceedings)  

DOI： 10.23919/APSIPA.2018.8659649

researchmap

Other Link： https://dblp.uni-trier.de/db/conf/apsipa/apsipa2018.html#MotonakaKKM18

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

We theoretically study the generalization capability of on-line learning using several correlated input vectors in each update in a statistical-mechanical manner. We consider a model organized with linear perceptrons with Gaussian noise. First, in a noiseless case, we analytically derive the optimal learning rate as a function of the number of examples used in one update and their correlation. Next, we analytically show that the use of correlated examples is effective if the optimal learning rate is used, even when there is some noise. Furthermore, we propose a novel algorithm that raises the generalization capability by increasing the number of examples used in one update with time.

DOI： 10.7566/JPSJ.86.084804

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

The generalization performance of semi-supervised learning is analyzed in the framework of online learning using the statistical-mechanical method. We derive deterministically formed simultaneous differential equations that describe the dynamical behaviors of order parameters using the self-averaging property under the thermodynamic limit. By generalizing the number of labeled data, the derived theory connects supervised learning and semi-supervised learning.

DOI： 10.7566/JPSJ.86.063801

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

We analyze the behaviors of the least-mean-square algorithm for a time-varying unknown system using a statistical-mechanical method. Cross-correlations between the elements of a primary path and those of an adaptive filter and autocorrelations of the elements of the adaptive filter are treated as macroscopic variables. We obtain simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables under conditions in which the tapped delay line is sufficiently long. We analytically show the existence of an optimal step size. This result is supporting evidence of Widrow et al.' s pioneering work that clarified the trade-off between the noise misadjustment and the lag misadjustment. Furthermore, we obtain the exact solution of the optimal step size in the case of a white reference signal. The derived theory includes the behaviors for a time-constant unknown system as a special case.

DOI： 10.7566/JPSJ.86.024803

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

Semi-supervised learning is a paradigm that uses a large number of unlabeled data and a small number of labeled data. We analyze the dynamical behaviors of semi-supervised learning in the framework of on-line learning by the statistical-mechanical method. A student uses several correlated input vectors in each update. The student is given a desired output for only one input vector out of these correlated input vectors. In this model, we derive simultaneous differential equations with deterministic forms that describe the dynamical behaviors of order parameters using the self-averaging property in the thermodynamic limit. We treat three well-known learning rules, that is, the Hebbian, Perceptron, and AdaTron learning rules. As a result, it is shown that using unlabeled data is effective in the early stages for all three learning rules. In addition, we show that the three learning rules have qualitatively different dynamical behaviors. Furthermore, we propose a new algorithm that improves the generalization performance by switching the number of input vectors used in an update as the time step proceeds.

DOI： 10.7566/JPSJ.85.084802

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We analyze the behaviors of the LMS algorithm for a time-varying unknown system using a statistical-mechanical method. We obtain simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables under conditions in which the tapped-delay line is sufficiently long. We show the existence of an optimal step size owing to the trade-off between the noise misadjustment and the lag misadjustment. Furthermore, we obtain the exact optimal step size in the case of a white reference signal. The derived theory includes the behaviors for a time-constant unknown system as a special case.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We analyze the behaviors of active noise control using a statistical-mechanical method. The principal assumption used in the analysis is that the impulse responses of the primary path and adaptive filter are sufficiently long. In particular, in this paper we analyze the adaptation rate of the mean square error (MSE) using two measures. The first measure is the MSE initial decreasing rate. The second measure is an adaptation constant. This is defined by the negative of the maximum eigenvalue of the coefficient matrix of differential equations that describe the dynamical behaviors of the macroscopic variables. Introducing these two measures, we theoretically show that the optimal step size depends on whether we focus on the rate of decrease in the MSE at the initial stage or the MSE after sufficient adaptation time.

Web of Science

researchmap

DOI： 10.5687/iscie.28.198

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

A theory that predicts the behaviors of the Filtered-X LMS algorithm was derived by using a statistical-mechanical method. In this paper, the theory is generalized to explain the system behaviors in the case of an actual primary path. In the theory, cross-correlations between the element of a primary path and that of an adaptive filter and autocorrelations of the elements of the adaptive filter are treated as macroscopic variables. Simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables are obtained under conditions in which the tapped-delay line is sufficiently long. The equations are analytically solved to obtain the correlations and finally compute the mean-square error. In order to generalize the theory to the case of an actual primary path, the correlations of the elements of the primary path are absorbed. The generalized theory quantitatively predict the behaviors in the case of an actual primary path.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We analyze the behaviors of active noise control with a time-varying primary path using a statistical-mechanical method. The principal assumption used in the analysis is that the impulse responses of the primary path and adaptive filter are sufficiently long. We analyze a novel model in which the reference signal is not necessarily white and the primary path is time-varying while its norm is kept constant in the mean sense. We show the existence of macroscopic steady states and the optimal step size.

Web of Science

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

DOI： 10.11316/jpsgaiyo.70.2.0_2850

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

DOI： 10.11316/jpsgaiyo.70.2.0_2849

CiNii Books

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

Since a model in which a student learns from two or more teachers who themselves are learning has a certain similarity with actual human society, the analysis of such a model is interesting. In this paper, a model composed of a true teacher, multiple moving ensemble teachers existing around the true teacher, and a student, which are all linear perceptions, is analyzed using the statistical-mechanical method in the framework of on-line learning. The dependences of the generalization performance on the ensemble teachers' learning rate, the student's learning rate, and the number of ensemble teachers are clarified. Furthermore, it is shown that the generalization error can be reduced to the lower bound in the case of moving ensemble teachers, while there are unattainable generalization errors in the case of stationary ensemble teachers. These results show that it is important for teachers to continue learning in order to educate students.

DOI： 10.7566/JPSJ.83.054801

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We analyze the learning curves of the active noise control with a time-varying primary path using a statistical mechanical method. The cross-correlation between the element of a primary path and that of the adaptive filter and the autocorrelations of the elements of the adaptive filter are treated as macroscopic variables. We obtain simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables under the condition that the tapped-delay line is sufficiently long. We analyze the case where the primary path has the Markovian property. As a result, we show that an optimal step size exists.

Web of Science

researchmap

Language：Japanese  

CiNii Books

researchmap

researchmap

Language：English   Publisher：The Physical Society of Japan  

DOI： 10.7566/JPSJ.82.023802

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper analyzed convergence properties of an online learning method when teacher's signal includes noise in the thermodynamic limit. The learning curve was analytically derived using a statistical mechanical method and its validity was confirmed by computer simulations. In this case, the learning curve shows an overshoot phenomenon. In order to elucidate why and how it occurs in this case, the asymptotic analysis of dynamical systems was applied to the differential equations that expresses the dynamics of the learning curve and showed that the phenomenon results from the properties of the system matrix of the equations. © Springer-Verlag 2013.

DOI： 10.1007/978-3-642-36669-7_51

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We analyze the learning curves of the FXLMS algorithm using a statistical-mechanical method when the reference signal is not necessarily white. We treat the nonwhite reference signal by introducing the correlation function of the signal to the method proposed in our previous study. Cross-correlations between the element of a primary path and that of an adaptive filter and autocorrelations of the elements of the adaptive filter are treated as macroscopic variables. We obtain simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables under the conditions in which the tapped-delay line is long. We analytically solve the equations to obtain the correlations and finally compute the mean-square error. The obtained theory quantitatively agrees with the results of computer simulations. The theory also gives the upper limit of the step size in the FXLMS algorithm.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We analyze the learning curves of the FXLMS algorithm using a statistical-mechanical method. Cross-correlations between the element of a primary path and that of an adaptive filter and autocorrelations of the elements of the adaptive filter are treated as macroscopic variables. We obtain simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables under the conditions in which the tapped-delay line is long. We analytically solve the equations to obtain the correlations and finally compute the mean-square error. Introducing the correlation function of the input signal, the theory can treat not only the white but also the nonwhite signal. The obtained theory quantitatively agrees with the results of computer simulations.

Web of Science

researchmap

researchmap

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We theoretically obtain the learning curves of the FXLMS algorithm on the basis of statistical mechanical analysis. Cosines of angles between the coefficient vectors of an adaptive filter, its shifted filters, and an unknown system are treated as macroscopic variables. Assuming that the tapped-delay line is sufficiently long and exactly calculating the correlations between the past tap input vectors and the coefficient vector of the adaptive filter, we obtain simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables in a deterministic form. We analytically solve the equations and show that the obtained theory quantitatively agrees with computer simulations. In the analysis, neither the independence assumption, the small step-size condition, nor the few-taps assumption is used.

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG  

Weight perturbation learning was proposed as a learning rule in which perturbation is added to the variable parameters of learning machines. The generalization performance of weight perturbation learning was analyzed by statistical mechanical methods and was found to have the same asymptotic generalization property as perceptron learning. In this paper we consider the difference between perceptron learning and AdaTron learning, both of which are well-known learning rules. By applying this difference to weight perturbation learning, we propose adaptive weight perturbation learning. The generalization performance of the proposed rule is analyzed by statistical mechanical methods, and it is shown that the pro: posed learning rule has an outstanding asymptotic property equivalent to that of AdaTron learning.

DOI： 10.1587/transinf.E94.D.1937

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG  

We consider a model composed of nonlinear perceptrons and analytically investigate its generalization performance using correlated examples in the framework of on-line learning by a statistical mechanical method. In Hebbian and AdaTron learning, the larger the number of examples used in an update, the slower the learning. In contrast, Perceptron learning does not exhibit such behaviors, and the learning becomes fast in some time region.

DOI： 10.1587/transinf.E94.D.1941

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

We derive a deterministic algorithm that restores and segments an image using belief propagation and a variational Bayesian method based on region-based latent variables and a coupled MRF model. This algorithm estimates two hyperparameters as well as infers the original image and the latent variables. In addition, the algorithm carries out model selection by minimizing the variational free energy. Through experiments using artificial images and a natural image degraded by Gaussian noises, we show that the derived algorithm has the potential ability to restore and segment using a single noisy image.

DOI： 10.1143/JPSJ.80.093802

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：INST ENGINEERING TECHNOLOGY-IET  

The learning curves of the filtered-X least-mean-square (LMS) algorithm are theoretically obtained using a statistical-mechanics approach. The direction cosines among the vectors of an adaptive filter, its shifted filters, and an unknown system are treated as macroscopic variables. Assuming that the tapped-delay line is sufficiently long, simultaneous differential equations are obtained that describe the dynamical behaviours of the macroscopic variables in a deterministic form. The equations are solved analytically and show that the obtained theory quantitatively agrees with computer simulations. In the analysis, neither the independence assumption nor the few-taps assumption is used.

DOI： 10.1049/el.2011.1691

Web of Science

researchmap

Language：Japanese   Publisher：The Institute of Electrical Engineers of Japan  

In this paper, we describe a combination of the particle swarm optimization (PSO) and the simultaneous perturbation optimization method. Moreover, FPGA implementation of this method is considered. Details of the hardware system are also explained.<br>PSO is an interesting optimization technique with powerful global search capability. PSO has plural search points as candidates of optima. However, this method does not utilize local information of an objective function such as gradient.<br>On the other hand, the simultaneous perturbation method is a recursive optimization method using a kind of finite difference. The method estimates gradient without direct calculation. Thus, we propose a combined method of PSO and the simultaneous perturbation method. This method has global search capability and local search one at the same time.

DOI： 10.1541/ieejeiss.131.682

CiNii Books

researchmap

Other Link： https://jlc.jst.go.jp/DN/JALC/00363913054?from=CiNii

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

To represent edges in image processing based on Bayesian inference, it is very effective to introduce latent variables. In this paper, we derive a deterministic algorithm that restores and segments an image using region-based latent variables and variational inference. This algorithm estimates two hyperparameters as well as infers the original image and the latent variables. In addition, the algorithm carries out model selection by minimizing the variational free energy. Through experiments using an artificial image generated by the heat bath method and natural images degraded by Gaussian noises, the effectiveness and limitations of the derived algorithm are shown.

DOI： 10.1143/JPSJ.80.014802

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER-VERLAG BERLIN  

In ensemble-teacher learning, a student learns from a quasi-optimal-teacher selected randomly from a pool of many quasi-optimal-teachers, and the student performs better than the quasi-optimal teachers after the learning. The student performance is improved by using many quasi-optimal-teachers when a Hebbian rule is used. However, a perceptron rule cannot improve the student performance. We previously proposed a novel ensemble-teacher learning using a perceptron rule with a margin. A perceptron rule with a margin is mid-way between a Hebbian rule and a perceptron rule. We have found through computer simulation that a perceptron rule with a margin can improve student performance. In this paper, we provide theoretical support to the proposed method by using statistical mechanics methods.

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG  

We propose an estimation of distribution algorithm in corporating switching The algorithm enables switching from the standard estimation of distribution algorithm (EDA) to the genetic algorithm (GA) or vice versa on the basis of switching criteria The algorithm shows better performance than GA and EDA in deceptive problems.

DOI： 10.1587/transinf.E93.D.3108

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：IOP PUBLISHING LTD  

We study a neural network model in which both neurons and synaptic interactions evolve in time simultaneously. The time evolution of synaptic interactions is described by a Langevin equation including a Hebbian learning term with the learning coefficient epsilon, and a bias term which is the interaction of the Hopfield model. We assume that synaptic interactions change is much slower than neurons and we study the stationary states of synaptic interactions by the replica method. We draw phase diagrams taking into account the stability of solutions, and find that the temperature region in which the Hopfield attractor is stable increases as the learning coefficient increases. Theoretical results are confirmed by the direct numerical integration of the Langevin equation. Further, we study the characteristics of the resultant synaptic interactions by partial annealing in the parameter region where the Hopfield and the mixed states exist. We find two kinds of interactions, one of which has the Hopfield attractor and the other has the mixed state attractor. Each interaction is characterized mainly by the eigenvector belonging to the largest eigenvalue of the interaction as a matrix.

DOI： 10.1088/1751-8113/43/2/025004

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

This paper discusses two types of vector classifiers for the hand posture recognition system. One classifier is based on the range check (RC) function, and the other is based on the hybrid network that is made of self-organizing map (SOM) and Hebbian network. In case the learning data and the testing data are different, the system with the hybrid network classifier outperforms the other system in the recognition rate by 9%. Two types of implementations are designed for the RC classifier. One uses parallel architecture and the other employs serial architecture. The size of the RC classifier in serial architecture is 38,000 gate count while the parallel architecture design requires 230,000 gate count. The circuit size of the hybrid network classifier is 606,000 gate count, even though the learning circuit is excluded in the design. The circuit size of the hybrid classifier is almost 2.6 times larger than that of the RC classifier, both use parallel architecture. Compared to the RC classifier in serial architecture, its size is 16 times bigger. Therefore the RC classifier is suitable for the hardware implementation even though the hybrid network classifier provides better performance.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER-VERLAG BERLIN  

Ensemble learning improves the performance of a learning machine by using a majority vote of many weak-learners. As an alternative, Miyoshi and Okada proposed ensemble-teacher learning. In this method, the student learns from many quasi-optimal teachers and performs better than the quasi-optimal teachers when a linear perceptron is used. When a non-linear perceptron is used, a Hebbian rule is effective; however, a perceptron rule is not effective in this case and the student cannot perform better than the quasi-optimal teachers. In this paper, we analyze ensemble-teacher learning and explain why a perceptron rule is not effective in ensemble-teacher learning. We propose a method to overcome this problem.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

This paper discusses the use of self-organizing map (SOM) in the image coding. The image is partitioned into blocks each of which is fed to the SOM as the training vectors. Then they are quantized into smaller number of vectors that is used as codeword to reconstruct the image. Thus the data size is reduced. In reconstructing the image, each block is filled with the corresponding codeword. The feasibility of the system is tested by computer simulations and the effect of the sizes of blocks and SOM on the compression rate and image quality is studied. Then the SOM based hardware image coding system is designed, and its functionality is also tested. It is estimated that the hardware SOM can code a single image within 1.2 ms.

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

We propose a new mutual learning using many weak learners (or students) which converges into the identical state of Bagging that is kind of ensemble learning, within the framework of on-line learning, and have analyzed its asymptotic property through the statistical mechanics method. Mutual learning involving three or more students fundamentally differs from the two-student case with regard to the variety of selecting a student to act as teacher. The proposed model consists of two learning steps: many students independently learn from a teacher, and then the students learn from others through the mutual learning. In mutual learning, students learn from other students and the generalization error is improved even if the teacher has not taken part in the mutual learning. We demonstrate that the learning style of selecting a student to act as teacher randomly is superior to that of cyclic order by using principle component analysis.

DOI： 10.1143/JPSJ.78.114001

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG  

We show that simultaneous perturbation can be used as an algorithm for on-line learning, and we report our theoretical investigation on generalization performance obtained with a statistical mechanical method. Asymptotic behavior of generalization error using this algorithm is on the order of t to the minus one-third power, where t is the learning time or the number of teaming examples. This order is the same as that using well-known perceptron learning.

DOI： 10.1587/transfun.E92.A.1743

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

We have analyzed the generalization performance of a student which slowly switches ensemble teachers. By calculating the generalization error analytically using statistical mechanics in the framework of on-line learning, we show that the dynamical behaviors of generalization error have a periodicity that is synchronized with the switching period and that the behaviors differ with the number of ensemble teachers. Furthermore, we show that the smaller the switching period is, the larger the difference in behavior is.

DOI： 10.1143/JPSJ.78.053001

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Self-organizing map (SOM) proposed by T. Kohonen is a neural network with unsupervised leaning to classify multi-dimensional vectors. The performance of SOM implemented in software decreases as the number of neurons increases. Therefore, performance acceleration of SOM by custom hardware is highly desired. In addition the hardware implementation can make the best use of the parallelism embedded in the SOM algorithm. VHSIC hardware description language (VHDL) is widely used to describe and design digital hardware but the VHDL description becomes larger in proportion to the size of SOM. This paper discusses the automatic generation of VHDL description of the hardware SOM by software. A hardware SOM generator is developed and its preliminary results are presented.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER-VERLAG BERLIN  

We analyzed the equilibrium states of an Ising spin neural network model in which both spins and interactions evolve simultaneously v over time. The interactions are Mexican-bat-type, which are used for lateral inhibition models. The model shows a bump activity, Which is the locally activated network state. The time-dependent interactions are driven by Langevin noise and Hebbian learning. The analysis results reveal that Hebbian learning expands the bistable regions of the ferromagnetic and local excitation phases.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

This paper discusses a hand posture recognition system with a hybrid network classifier. The hybrid network consists of SOM and Hebbian network. Feature vector is extracted from the input hand posture image and the given feature vector is mapped to a lower-dimensional map by the SOM. Then the supervised Hebbian network performs category acquisition and naming. The feasibility of the system is verified by computer simulations. The results show that the recognition performance of the system is quite good if the number of neurons in the SOM is sufficient. Besides the recognition performance, the advantage of the hybrid classifier is the embedded learning capability. It is also expected that the classifier can be extended to recognize dynamic gesture by employing feedback SOM.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：SPRINGER-VERLAG BERLIN  

We propose a new mutual learning using many weak learner (or student) which converges into the identical state of Bagging that is kind of ensemble learning, within the framework of on-line learning, and have analyzed its asymptotic property through the statistical mechanics method. Mutual learning involving more than three students is essential compares to two student case from a viewpoint of variety of selection of a student acting as teacher. The proposed model consists of two learning steps: many students independently learn from a teacher, and then the students learn from others through the mutual learning. In mutual learning, students learn from other students and the generalization error is improved even if the teacher has not taken part in the mutual learning. We demonstrate that the learning style of selecting a student to act as teacher randomly is superior to that of cyclic order by using principle component analysis.

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG  

In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of on-line learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.

DOI： 10.1093/ietfec/e91-a.9.2663

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

We analyze the generalization performance of a student in a model composed of nonlinear perceptrons: a true teacher, ensemble teachers, and the student. We calculate the generalization error of the student analytically or numerically using statistical mechanics in the framework of on-line learning. We treat two well-known learning rules: Hebbian learning and perceptron learning. As a result, it is proven that the nonlinear model shows qualitatively different behaviors from the linear model. Moreover, it is clarified that Hebbian learning and perceptron learning show qualitatively different behaviors from each other. In Hebbian learning, we can analytically obtain the solutions. In this case, the generalization error monotonically decreases. The steady value of the generalization error is independent of the learning rate. The larger the number of teachers is and the more variety the ensemble teachers have, the smaller the generalization error is. In perceptron learning, we have to numerically obtain the solutions. In this case, the dynamical behaviors of the generalization error are nonmonotonic. The smaller the learning rate is, the larger the number of teachers is; and the more variety the ensemble teachers have, the smaller the minimum value of the generalization error is.

DOI： 10.1143/JPSJ.76.114001

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

We analyze the time domain ensemble on-line learning of a Perceptron under the existence of external noise. We adopt three typical learning rules, Hebbian, Perceptron, and AdaTron rules. We treat the input and output noises. In order to improve the learning when it does not succeed in the sense that the student vector does not converge to the teacher vector, we use an averaging method and give theoretical analysis of the method. We obtain the precise formula for the overlap between the teacher vector and the time averaged student vector for t -&gt; infinity limit as a function of the number of student vectors to be averaged. We compare the theoretical results with numerical simulations and find that the theoretical results agree quite well with the numerical simulations.

DOI： 10.1143/JPSJ.76.114006

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

in the framework of on-line learning, a learning machine might move around a teacher due to the differences in structures or output functions between the teacher and the learning machine. In this paper we analyze the generalization performance of a new student supervised by a moving machine. A model composed of a fixed true teacher, a moving teacher, and a student is treated theoretically using statistical mechanics, where the true teacher is a nonmonotonic perceptron and the others are simple perceptrons. Calculating the generalization errors numerically, we show that the generalization errors of a student can temporarily become smaller than that of a moving teacher and can reach the lowest value, even if the student only uses examples from the moving teacher. However, the generalization error of the student eventually becomes the same value with that of the moving teacher. This behavior is qualitatively different from that of a linear model.

DOI： 10.1143/JPSJ.76.044003

Web of Science

researchmap

Language：English   Publisher：The Physical Society of Japan (JPS)  

Grant-in-Aid for Scientific Research

DOI： 10.1143/JPSJ.75.124002

CiNii Books

researchmap

Other Link： https://jlc.jst.go.jp/DN/JALC/00335889653?from=CiNii

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

Conventional ensemble learning combines students in the space domain. On the other hand, in this paper, we combine students in the time domain and call it time-domain ensemble learning. We analyze the generalization performance of time-domain ensemble learning in the framework of on-line learning using a statistical mechanical method. We use a model in which both the teacher and the student are linear perceptrons with noises. Time-domain ensemble learning is twice as effective as conventional space-domain ensemble learning.

DOI： 10.1143/JPSJ.75.084007

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

We analyze the generalization performance of a student in a model composed of linear perceptrons: a true teacher, ensemble teachers, and the student. Calculating the generalization error of the student analytically using statistical mechanics in the framework of on-line learning, we prove that when the learning rate satisfies eta &lt; 1, the larger the number K is and the more variety the ensemble teachers have, the smaller the generalization error is. On the other hand, when eta &gt; 1, the properties are completely reversed. If the variety of the ensemble teachers is rich enough, the direction cosine between the true teacher and the student becomes unity in the limit of eta -&gt; 0 and K -&gt; infinity. Intuitive interpretations of these results are given.

DOI： 10.1143/JPSJ.75.044002

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PHYSICAL SOC JAPAN  

In the framework of on-line learning, a learning machine might move around a teacher due to the differences in structures or output functions between the teacher and the learning machine or due to noises. The generalization performance of a new student supervised by a moving machine has been analyzed. A model composed of a fixed true teacher, a moving teacher and a student that are all linear perceptrons with noises has been treated analytically using statistical mechanics. It has been proven that the generalization errors of a student can be smaller than that of a moving teacher, even if the student only uses examples from the moving teacher.

DOI： 10.1143/JPSJ.75.024003

Web of Science

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

CiNii Books

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

We analyze the generalization performance of a student in a model composed of linear perceptrons: a true teacher, ensemble teachers, and the student. Calculating the generalization error of the student analytically using statistical mechanics in the framework of online learning, we prove that when the learning rate satisfies eta &lt; 1, the larger the number K is and the more variety the ensemble teachers have, the smaller the generalization error is. On the other hand, when eta &gt; 1, the properties are completely reversed. If the variety of the ensemble teachers is rich enough, the direction cosine between the true teacher and the student becomes unity in the limit of eta -&gt; 0 and K -&gt; infinity.

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：Institute of Electrical and Electronics Engineers Inc.  

We analyze the generalization performance of a student in a model composed of linear perceptrons: a true teacher, ensemble teachers, and the student. Calculating the generalization error of the student analytically using statistical mechanics in the framework of online learning, we prove that when the learning rate satisfies η &lt
1, the larger the number K is and the more variety the ensemble teachers have, the smaller the generalization error is. On the other hand, when η &gt
1, the properties are completely reversed. If the variety of the ensemble teachers is rich enough, the direction cosine between the true teacher and the student becomes unity in the limit of η → 0 and K → ∞. ©2006 IEEE.

DOI： 10.1109/ijcnn.2006.246759

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)  

We discuss ensemble learning using K simple non-linear perceptrons and a sign output function within the framework of statistical mechanics of on-line learning. One objective of statistical learning theory is to calculate the theoretical generalization error of a learner. In this paper we show that the generalization error of K students when their combined output is determined by a majority vote can be calculated as a function of two macroscopic variables: the degree of similarity between the teacher and the students, and the degree of similarity among the students. We then derive differential equations describing the dynamics of these macroscopic variables for a generalized learning rule. In addition, we derive these differential equations explicitly for three learning rules: the well-known Hebbian learning, Perceptron learning, and AdaTron learning rules. Solving these, we calculate the generalization error for each in turn numerically. As a result we find that these three learning rules have different characteristics with respect to their affinity to ensemble learning as seen in their conservation of diversity among students
and the fascinating fact that from this perspective the AdaTron learning rule is superior is brought to light. © 2005 Wiley Periodicals, Inc.

DOI： 10.1002/scj.20336

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：AMERICAN PHYSICAL SOC  

Ensemble learning of K nonlinear perceptrons, which determine their outputs by sign functions, is discussed within the framework of online learning and statistical mechanics. One purpose of statistical learning theory is to theoretically obtain the generalization error. This paper shows that ensemble generalization error can be calculated by using two order parameters, that is, the similarity between a teacher and a student, and the similarity among students. The differential equations that describe the dynamical behaviors of these order parameters are derived in the case of general learning rules. The concrete forms of these differential equations are derived analytically in the cases of three well-known rules: Hebbian learning, perceptron learning, and AdaTron (adaptive perceptron) learning. Ensemble generalization errors of these three rules are calculated by using the results determined by solving their differential equations. As a result, these three rules show different characteristics in their affinity for ensemble learning, that is "maintaining variety among students." Results show that AdaTron learning is superior to the other two rules with respect to that affinity.

DOI： 10.1103/PhysRevE.71.036116

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)  

Ensemble learning of K nonlinear perceptrons, which determine their outputs by sign functions, is discussed within the framework of online learning and statistical mechanics. One purpose of statistical learning theory is to theoretically obtain the generalization error. This paper shows that ensemble generalization error can be calculated by using two order parameters, that is, the similarity between a teacher and a student, and the similarity among students. The differential equations that describe the dynamical behaviors of these order parameters are derived in the case of general learning rules. The concrete forms of these differential equations are derived analytically in the cases of three well-known rules: Hebbian learning, perceptron learning, and AdaTron (adaptive perceptron) learning. Ensemble generalization errors of these three rules are calculated by using the results determined by solving their differential equations. As a result, these three rules show different characteristics in their affinity for ensemble learning, that is "maintaining variety among students." Results show that AdaTron learning is superior to the other two rules with respect to that affinity. ©2005 The American Physical Society.

DOI： 10.1103/PhysRevE.71.036116

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1143/PTPS.157.270

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC  

It is known that storage capacity per synapse increases by synaptic pruning in the case of a correlation-type associative memory model. However, the storage capacity of the entire network then decreases. To overcome this difficulty, we propose decreasing the connectivity while keeping the total number of synapses constant by introducing delayed synapses. In this paper, a discrete synchronous-type model with both delayed synapses and their prunings is discussed as a concrete example of the proposal. First, we explain the Yanai-Kim theory by employing statistical neurodynamics. This theory involves macrodynamical equations for the dynamics of a network with serial delay elements. Next, considering the translational symmetry of the explained equations, we rederive macroscopic steady-state equations of the model by using the discrete Fourier transformation. The storage capacities are analyzed quantitatively. Furthermore, two types of synaptic prunings are treated analytically: random pruning and systematic pruning. As a result, it becomes clear that in both prunings, the storage capacity increases as the length of delay increases and the connectivity of the synapses decreases when the total number of synapses is constant. Moreover, an interesting fact becomes clear: the storage capacity asymptotically approaches 2/pi due to random pruning. In contrast, the storage capacity diverges in proportion to the logarithm of the length of delay by systematic pruning and the proportion constant is 4/pi. These results theoretically support the significance of pruning following an overgrowth of synapses in the brain and may suggest that the brain prefers to store dynamic attractors such as sequences and limit cycles rather than equilibrium states.

DOI： 10.1109/TNN.2004.832711

Web of Science

researchmap

科研費基盤研究

researchmap

Language：English   Publishing type：Research paper (scientific journal)   Publisher：PERGAMON-ELSEVIER SCIENCE LTD  

The synapses of real neural systems seem to have delays. Therefore, it is worthwhile to analyze associative memory models with delayed synapses. Thus, a sequential associative memory model with delayed synapses is discussed, where a discrete synchronous updating rule and a correlation learning rule are employed. Its dynamic properties are analyzed by the statistical neurodynamics. In this paper, we first re-derive the Yanai-Kim theory, which involves macrodynamical equations for the dynamics of the network with serial delay elements. Since their theory needs a computational complexity of O(L(4)t) to obtain the macroscopic state at time step I where L is the length of delay, it is intractable to discuss the macroscopic properties for a large L limit. Thus, we derive steady state equations using the discrete Fourier transformation, where the computational complexity does not formally depend on L. We show that the storage capacity alpha(C) is in proportion to the delay length L with a large L limit, and the proportion constant is 0.195, i.e. alpha(C) = 0.195L. These results are supported by computer simulations. (C) 2003 Elsevier Ltd. All rights reserved.

DOI： 10.1016/S0893-6080(03)00207-7

Web of Science

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：IEEE  

Ensemble learning of K nonlinear perceptrons, which determine their outputs by sign functions, is discussed within the framework of online learning and statistical mechanics. This paper shows that ensemble generalization error can be calculated by using two order parameters, that is, the similarity between a teacher and a student, and the similarity among students. The differential equations that describe the dynamical behaviors of these parameters are derived analytically in the cases of Hebbian, perceptron and AdaTron learning. These three rules show different characteristics in their affinity for ensemble learning, that is "maintaining variety among students." Results show that AdaTron learning is superior to the other two rules.

Web of Science

researchmap

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1002/ecjc.10037

researchmap

科研費奨励研究

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：NANYANG TECHNOLOGICAL UNIV  

The synapses of real neural systems seem to have delays. Therefore, it is worthwhile to analyze associative memory models with delayed synapses. Thus, a sequential associative memory model with delayed synapses is discussed, where a discrete synchronous updating rule and a correlation learning rule are employed. Its dynamic properties are analyzed by the statistical neurodynamics. In this paper, we first re-derive the Yanai-Kim theory, which involves macrodynamical equations for the dynamics of the network with serial delay elements. Since their theory needs a computational complexity of O(L(4)t) to obtain the macroscopic state at time step t where L is the length of delay, it is intractable to discuss the macroscopic properties for a large L limit. Thus, we derive steady state equations using the discrete Fourier transformation, where the computational complexity does not formally depend on L. We show that the storage capacity alpha(C) is in proportion to the delay length L with a large L limit, and the proportion constant is 0.195, i.e., alpha(C) = 0.195L. These results are supported by computer simulations.

Web of Science

researchmap

researchmap

DOI： 10.1002/(SICI)1520-6440(199909)82:9<29::AID-ECJC4>3.0.CO;2-P

researchmap

Other Link： http://orcid.org/0000-0003-3330-6121

researchmap

Other Link： http://orcid.org/0000-0003-3330-6121

researchmap

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)   Publisher：OHMSHA LTD  

The block orthogonal projection algorithm which is one for transversal filters can be applied to a linear dichotomy (so called Perceptron) which consists of a transversal filter and a sign function. When the block size which is the number of examples used in one time of renewal is one, the algorithm is equivalent to the normalized LMS algorithm and is proven to stop in a finite number of iterations when the learning coefficient is unity. This report gives the block size which maximizes the convergence rate when the learning coefficient is unity, and confirms it by computer simulations. The results say that larger block size is not necessarily better.

Web of Science

researchmap

researchmap

Other Link： http://orcid.org/0000-0003-3330-6121

researchmap

Language：Japanese   Publisher：電気学会  

CiNii Books

researchmap

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

CiNii Books

researchmap

researchmap

researchmap

researchmap

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

We analyze the dynamical behaviors (learning curves) of the active noise control using a statistical-method. Cross-correlation between a primary path and an adaptive filter are treated as macroscopic variable. By taking the correlations between past tap input vectors and the coefficient vector of the adaptive filter into consideration, we obtain simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables under the condition in which the tapped-delay line is sufficiently long. In this report, we generalize and relax model restriction of a primary path in an earlier report. The obtained theory quantitatively agrees with the results of computer simulations.

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese  

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

Code Division Multiple Access (CDMA) is a technique for multiple access, and used for mobile phone, satellite communications and so on. In Binary Phase Shift keying (BPSK) the performance of multiuser detection based on posterior destribution has been analyzed. However, in the field of digital communications, the case of Quaternary PSK (QPSK) in which the user data signal is quaternary or of 8PSK in which the user data signal is octal has also been used. Therefore we analyze the performance of CDMA multiuser detection with M-ary phase-shift keying based on posterior by using the replica method.

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

We analyze the dynamical behaviors (learning curves) of active noise control with FXLMS algorithm using statistical mechanical method. The cross-correlation between an unknown system and an adaptive filter and autocorrelation of the adaptive filter are treated as the macroscopic variables. We obtain the simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables under the conditions in which the reference signal is white and the tapped-delay line is long. We analytically solve the equations. Neither the independence assumption, the sinusoidal input assumption, the small step-size condition, nor the few-taps assumption is used. We discuss the systematic behaviors and adaptation rate using the derived theory and compare the theory with the simulation results using the real impulse response data of the primary path measured in the laboratory.

CiNii Books

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese  

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

We derive a deterministic algorithm that restores and segments an image using belief propagation and a variational Bayesian method based on region-based latent variables and a coupled MRF model. This algorithm estimates two hyperparameters as well as infers the original image and the latent variables. In addition, the algorithm carries out model selection by minimizing the variational free energy. Through experiments using an artificial image and a natural image degraded by Gaussian noises, we show that the derived algorithm has the potential ability to restore and segment using a single noisy image.

CiNii Books

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

We analyze the dynamical behaviors (learning curves) of active noise control with Filtered-X LMS algorithm using statistical mechanical method. Direction cosines among coefficient vectors of an adaptive filter, its shifted filters, and an unknown system are treated as the macroscopic variables. We obtain the simultaneous differential equations that describe the dynamical behaviors of the macroscopic variables when the tapped-delay line is long. We analytically solve the equations. Neither the independence assumption nor the small step sizes condition are used. The obtained theory quantitatively agrees with computer simulations, regardless of whether there is an error in the secondary path estimation.

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

Publisher：社団法人電子情報通信学会  

ベイズ推定を用いる画像処理においてエッジを表現するためには隠れ変数の導入が有効である.本稿では,領域ベースの隠れ変数を用いる結合MRFモデルに基づき,確率伝搬法の一種であるビリーフプロパゲーションと変分推論法を組み合わせた手法を用いて画像の修復と領域分割を行う決定論的なアルゴリズムを導出する.このアルゴリズムでは原画像や隠れ変数だけでなく2個のハイパーパラメータも推定する.さらに,変分自由エネルギー最小化によるモデル選択も行う.熱浴法で生成した人工画像やガウス雑音が重畳された自然画像を用いた実験により,提案手法が一枚の劣化画像だけから良好な修復と領域分割を行う潜在能力を有することを示す.

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

The weight perturbation learning was proposed as a learning rule which adds perturbation to the variable parameters of learning machines. Generalization performance of the weight perturbation learning was analyzed by statistical mechanical methods. The weight perturbation learning has the same asymptotic generalization property as the Perceptron learning. In this paper we consider difference between the Perceptron learning and the AdaTron learning which are well-known learning rules. Applying the consideration to the weight perturbation learning, we propose the adaptive weight perturbation learning. The generalization performance of the proposed rule is analyzed by statistical mechanical methods. Consequently, it is shown that the proposed learning rule has an outstanding asymptotic property corresponding to the AdaTron learning.

CiNii Books

researchmap

Publisher：社団法人電子情報通信学会  

二次元画像から三次元世界を再構成することは画像工学と視覚脳科学の共通の目的の一つであり,画像の領域分割はこの再構成の手掛かりとなる重要な,そして難しい問題である.本稿では,領域分割が画像の認識や理解のための単なる前処理ではないという観点に立ち,領域ベースの隠れ変数を用いた結合マルコフ確率場に基づくベイズ推定により画像の修復と領域分割を行うアルゴリズムを導出する.このとき,解析計算や数値計算が困難となるので近似解析手法の一種である変分推論法を用いる.更に,結合マルコフ確率場と脳の視覚情報処理の関係についても述べる.

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：Kansai University  

CiNii Books

researchmap

Other Link： http://hdl.handle.net/10112/848

researchmap

Publisher：社団法人電子情報通信学会  

本論文ではイジングスピンニューラルネットワークおいて,相互作用とニューロンの両者がダイナミクスを持つ系の平衡状態を解析した.相互作用は側抑制モデルで用いられるメキシカンハット型であり,孤立局在興奮性の挙動を示す.また,相互作用はランジュバンノイズおよびヘブ則に似た学習規則によって変化する.解析の結果,ヘブ則の効果を強くすると,孤立局在と強磁性の混在する双安定領域が拡大することが分かった.

researchmap

Publisher：社団法人日本物理学会  

researchmap

Publisher：社団法人日本物理学会  

researchmap

Publisher：社団法人日本物理学会  

researchmap

Publisher：社団法人日本物理学会  

researchmap

Language：Japanese   Publisher：Kobe City College of Technology  

We treat the two-body Sourlas codes as an example to investigate the potential of double dynamics in the field of information technology. The error-correcting properties of the two-body Sourlas codes, in which the partial annealing is applied to the interaction system for decoding, are analyzed through the replica method. When the interactions are changed by Hebbian rule of the spins, an overlap of information bits and decoded bits becomes flat in the wide range of an inverse temperature of the spins.

CiNii Books

researchmap

Language：Japanese   Publisher：Kobe City College of Technology  

Learning is to infer the underlying rules that dominate data generation using observed data. In this paper we theoretically obtain the generalization error of a model in which both a student and a teacher are linear perceptrons using the statistical mechanical method in the framework of on-line learning. In case of a gradient method, the condition which the learning rate should satisfy becomes strict as the correlation among inputs or the number of inputs used in an update becomes large. We propose block orthogonal projection learning rule as a new algorithm which in not effected by the correlation. The properties are analytically clarified.

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Language：Japanese   Publisher：Institute of Systems, Control and Information Engineers  

CiNii Books

researchmap

Language：Japanese   Publisher：Kobe City College of Technology  

Learning is to infer the underlying rules that dominate data generation using observed data. In this paper we theoretically obtain the generalization error of a model in which both a student and a teacher are linear perceptrons using the statistical mechanical method in the framework of on-line learning. In case of a gradient method, the condition which the learning rate should satisfy becomes strict as the correlation among inputs or the number of inputs used in an update becomes large. In addition, we propose the new algorithm which is not effected by the correlation. The properties are investigated through computer simulations.

CiNii Books

researchmap

Language：Japanese   Publisher：The Physical Society of Japan (JPS)  

CiNii Books

researchmap

Publisher：社団法人日本物理学会  

researchmap

Publisher：社団法人日本物理学会  

researchmap

Publisher：社団法人日本物理学会  

researchmap

researchmap

researchmap

researchmap

Publisher：社団法人日本物理学会  

researchmap

researchmap

Publisher：社団法人電子情報通信学会  

オンライン学習において, 教師と生徒の構造や出力特性の相違, 雑音の影響などにより汎化誤差がゼロにならないモデルでは, 学習機械が真の教師のまわりを動き続ける場合がある.この動き続ける学習機械を教師とするような新たな生徒を考えこの生徒が真の教師に対してどれほどの汎化能力を持つことができるかを解析した.真の教師, 動く教師, 生徒のいずれもが雑音の重畳された線形なパーセプトロンであるモデルについて, 統計力学的手法により汎化誤差を解析的に求めた結果, 生徒が真の教師の入出力ではなく, 動く教師の入出力だけを例題として使用するにもかかわらず, 真の教師と動く教師の汎化誤差よりも真の教師と生徒の汎化誤差の方が小さくなりうることが明らかになった.

researchmap

Publisher：社団法人電子情報通信学会  

アンサンブル学習の大きな特徴として, 多数決などで生徒を組み合わせることにより, 単一の生徒では表現できない入出力関係を実現できることがあげられる.その意味で, 教師が生徒一個のモデル空間内にないような場合のアンサンブル学習の解析は非常に興味深い.そこで本論文では, 教師が非単調なパーセプトロンであり生徒が単純パーセプトロンである場合のアンサンブル学習を統計力学的なオンライン学習の枠組みで議論する.メトロポリス法により汎化誤差を計算した結果, ヘブ学習では学習の初期においてアンサンブルの効果があるものの, やがて生徒の多様性がなくなりアンサンブルの効果も消滅してしまうことがわかった.これに対し, パーセプトロン学習では生徒の多様性は消滅せず, そのために十分時間が経過した後でもアンサンブルの効果が残ることがわかった.

researchmap

Publisher：社団法人日本物理学会  

researchmap

researchmap

researchmap

Publisher：社団法人電子情報通信学会  

アンサンブル学習の大きな特徴として,多数決などで生徒を組み合わせることにより,単一の生徒では表現できない入出力関係を実現できることがあげられる,その意味で,教師が生徒一個のモデル空間内にないような場合のアンサンブル学習の解析は非常に興味深い.そこで本論文では,教師がコミティマシンであり生徒が単純パーセプトロンである場合のアンサンブル学習を統計力学的なオンライン学習の枠組みで議論する.メトロポリス法により汎化誤差を計算した結果,ヘブ学習ではすべての生徒は教師中間層の中央に漸近すること,パーセプトロン学習では生徒の多様性が消滅せず,そのためにアンサンブルの効果が残ること,アダトロン学習では一種の過学習が起こることなど,学習則毎の顕著な特徴が明らかになった.

researchmap

Publisher：社団法人日本物理学会  

researchmap

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

We discuss the ensemble learning using K nonlinear simple perceptrons of which an output function is the sign function based on the on-line learning in the finite K case. First, we derive a macroscopic differential equation describing a dynamics of correlation q between the student weight vectors in a general learning algorithm. Second, we apply the equation to the three well-known rules, that is the Hebb rule, the Perceptron rule and the AdaTron rule, and solve those numerically. Third, we obtain the generalization error of these ensemble machines using a majority vote of students. As result, we show that the correlation between the student weight vectors in the AdaTron rule evolves most slowly, and that the AdaTron rule is the most superior among the three learning rules in the framework of the ensemble learning.

CiNii Books

researchmap

Publisher：社団法人電子情報通信学会  

本講演では,バギングやパラレルブースティングなどのアンサンブル学習を統計力学的なオンライン学習の枠組で議論する.教師と生徒は両方とも単純パーセプトロンである場合を議論する.アンサンブル学習機械の汎化誤差が生徒の結合荷重ベクトルの大きさ,教師と生徒の荷重ベクトルのオーバーラップ(方向余弦)および生徒の結合荷重ベクトル間のオーバラップ(相関)にのみ依存することを示す.これらの巨視的な変数の学習のダイナミクスを記述する方程式を導出し,バギングとパラレルブースティングの性質を議論する.

researchmap

researchmap

researchmap

researchmap

Language：Japanese   Publisher：神戸市立工業高等専門学校  

CiNii Books

researchmap

Language：Japanese   Publisher：Kobe City College of Technology  

The geometric learning algorithm (GLA) and the symmetric learning algorithm (SLA) have been proposed for pattern classification by an elementary perceptron. In the case of kth order GLA and SLA, the connection weight vectors are updated vertically towards the orthogonal complement of k patterns to be classified. Therefore, it can be said that they are the block orthogonal projection learning algorithms. Moreover, the first order GLA equals to the AdaTron learning. In this paper, the offset block orthogonal projection learning algorithm (OBOP) is proposed for learning acceleration and its convergence properties are analyzed numerically. Furthermore, a plan of the theoretical analysis of these algorithms in the case of On-line learning is described.

CiNii Books

researchmap

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

CiNii Books

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

The geometric learning algorithm(GLA)and the symmetric learning algorithm(SLA)were proposed for an elementary perceptron. They are applications of the affine projection algorithm(APA)which is the algorithm for adaptive filters. In the case of κth order GLA and SLA, the connection weight vectors are updated vertically towards the orthogonal complement of κ patterns to be classified. Therefore, it can be said that they are the block orthogonal projection learning algorithms. In this report, the block orthogonal projection learning algorithm is modified and applied to multilayer neural networks, that is, the BOPM. The convergence properties of the BOPM are investigated through computer simulation. By selecting the order appropriately, the BOPM has the possibilities to realize faster learning than the basic back error propagation.

CiNii Books

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

The block orthogonal projection algorithm which is one for transversal filters can be applied to the linear dichotomy (what is called the perceptron) which is nonlinear. When the block size which is the number of examples used in one renewal is one, the algorithm is equivalent to the normalized LMS algorithm and is proven to stop in a finite number of iterations when the learning rate is unity. This report gives the block size which maximizes the convergence speed when the learning rate is unity, and confirms it by computer simulations. The results say that larger block size is not necessarily better.

CiNii Books

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

Language：Japanese   Publisher：The Institute of Electronics, Information and Communication Engineers  

The geometric learning algorithm (GLA) is proposed for an elementary perceptron. The GLA is a modified version of the affine projection algorithm (APA) for adaptive filters. The convergence conditions of the APA and the GLA are different. The convergence condition of the 1st order GLA for 2 patterns is theoretically derived. The new oncept "the angle of the solution area" is introduced. The computer simulation results support that this new concept is a good estimation of the convergence properties. The noise performance of the 1st order GLA is also analyzed.

CiNii Books

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

Language：Japanese   Publisher：Kobe City College of Technology  

A recurrent neural network(RNN), in which each unit has serial delay elements, is proposed for memorizing limit cycles(LCs). This network is called DRNN in this paper. An LC consists of several basic patterns. The hysteresis information of LCs, realized on the connections from the delay elements to the units, is very efficient in the following reasons. First, the same basic patterns can be shared by different LCs. This make it possible to drastically increase the number of LCs, even though using a small number of the basic patterns. Second, noise performance, that is, probability of recalling the exact LC starting from the noisy LC, can be improved. The hysteresis information consists of two components, the order of the basic patterns included in an LC, and the cross-correlation among all the basic patterns. In order to achieve good noise performance, a small number of the basic patterns is preferred. These properties of the DRNN are theoretically analyzed and confirmed through computer simulations. It is also confirmed that the DRNN is superior to the RNN without delay elements for memorizing LCs.

CiNii Books

researchmap

Language：English   Publisher：I E E E  

Web of Science

researchmap

researchmap

researchmap

researchmap

researchmap

Venue：Honolulu, Hawaii  

researchmap

researchmap

researchmap

Venue：Kuala Lumpur, Malaysia  

researchmap

Venue：Jeju, Korea  

researchmap

Venue：Jeju, Korea  

researchmap

Venue：大阪府立大学  

researchmap

Venue：関西大学  

researchmap

Venue：関西大学  

researchmap

Venue：Kansai University, Japan  

researchmap

Venue：Kansai University, Japan  

researchmap

Venue：Kansai University, Japan  

researchmap

Venue：Kansai University, Japan  

researchmap

Venue：Kansai University, Japan  

researchmap

Venue：Kansai University, Japan  

researchmap

Venue：Palais des Congres Lyon, France  

researchmap

Venue：九州大学  

researchmap

Venue：九州大学  

researchmap

Venue：Hong Kong  

researchmap

researchmap

Venue：福島  

researchmap

Venue：福島  

researchmap

Venue：関西大学  

researchmap

Venue：関西大学  

researchmap

Venue：Brisbane, Australia  

researchmap

Venue：Siem Reap, city of Angkor Wat, Cambodia  

researchmap

Venue：奈良先端科学技術大学院大学  

researchmap

Venue：京都  

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

Venue：Trieste, Italy  

researchmap

researchmap

Venue：Kansai University, Japan  

researchmap

researchmap

Venue：Kansai University, Japan  

researchmap

Venue：Vancouver, Canada  

researchmap

Venue：関西大学  

researchmap

Venue：関西大学  

researchmap

Venue：関西大学  

researchmap

Venue：関西大学  

researchmap

Venue：関西大学  

researchmap

Venue：石垣島  

researchmap

Venue：筑波大学東京キャンパス  

researchmap

Venue：筑波大学東京キャンパス  

researchmap

Venue：Nanjing, China  

researchmap

Venue：横浜国立大学  

researchmap

researchmap

Venue：Pulau Pinang, Malaysia  

researchmap

Venue：Pulau Pinang, Malaysia  

researchmap

researchmap

Venue：Ann Arbor, USA  

researchmap

Venue：大阪大学  

researchmap

Venue：関西学院大学  

researchmap

Venue：関西学院大学  

researchmap

Venue：関西学院大学  

researchmap

Venue：関西学院大学  

researchmap

Venue：岡山大学  

researchmap

researchmap

Venue：札幌コンベンションセンター  

researchmap

Venue：札幌コンベンションセンター  

researchmap

Venue：奈良女子大学  

researchmap

Venue：奈良女子大学  

researchmap

Venue：奈良女子大学  

researchmap

Venue：奈良女子大学  

researchmap

Venue：兵庫県立大学  

researchmap

Venue：兵庫県立大学  

researchmap

Venue：兵庫県立大学  

researchmap

Venue：兵庫県立大学  

researchmap

Venue：兵庫県立大学  

researchmap

Venue：兵庫県立大学  

researchmap

Venue：富山大学  

researchmap

Venue：富山大学  

researchmap

Venue：神戸大  

researchmap

Venue：神戸大  

researchmap

Venue：金沢市文化ホール  

researchmap

Venue：沖縄県青年会館  

researchmap

Venue：東大 武田ホール  

researchmap

Venue：Espoo, Finland  

researchmap

Venue：立命館大学大阪キャンパス  

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

Venue：Thessaloniki, Greece  

researchmap

Venue：Taipei, Taiwan  

researchmap

Venue：Taipei, Taiwan  

researchmap

researchmap

Venue：Barcelona, Spain  

researchmap

Venue：Barcelona, Spain  

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

原稿4pages， poster

researchmap

poster

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

pdf,poster

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

slide

researchmap

researchmap

researchmap

researchmap

slide

researchmap

researchmap

researchmap

researchmap

pdf, poster

researchmap

researchmap

researchmap

researchmap

pdf, slide

researchmap

pdf, poster

researchmap

Grant-in-Aid for Scientific Research

researchmap

pdf, poster

researchmap

Grant-in-Aid for Scientific Research

researchmap

slide

researchmap

Grant-in-Aid for Scientific Research

researchmap

pdf, poster

researchmap

pdf, poster

researchmap

pdf, slide

researchmap

Grant-in-Aid for Scientific Research

researchmap

slide

researchmap

pdf, slide

researchmap

pdf, slide

researchmap

pdf, slide

researchmap

researchmap

Grant-in-Aid for Scientific Research

researchmap

researchmap

researchmap

pdf, poster

researchmap

pdf、OHP

researchmap

pdf, poster

researchmap

pdf

researchmap

pdf, slide

researchmap

Grant-in-Aid for Encouragement of Young Scientists

researchmap

researchmap

researchmap

Grant-in-Aid for Encouragement of Young Scientists

researchmap

講演論文概要集，p.58

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap

researchmap