For instance max cut is a special case even when the function is required only to be nonnegative. Fast semidifferential based submodular function optimization. The resulting sketch is an approximately submodular function. Submodular functions are relevant to machine learning for mainly two reasons. You may be thinking you dont have to be a genius to make that inference so ill expand a little. Submodular programs constrained optimization 228 a lagrangian functions and optimality conditions 229 b related problems 234 b. Yet, there are several interesting cases which are in general not dr submodular, but can still be captured by a generalized notion.
In doing so, we uncover several novel structural results. In this paper, we rst describe tight modular bounds on submodular functions in section2, including lower. Optimal bounds on approximation of submodular and xos. Application for learning influence fnc in diffusion networks. The adaptive complexity of maximizing a submodular function. Learnability, structure, and optimization submodular functions are discrete functions that model laws of diminishing returns and enjoy numerous. Optimizing decomposable submodular functions institute.
Submodular optimization and approximation algorithms satoru iwata rims, kyoto university submodular functions cut capacity functions matroid rank functions entropy functions finite set. An analysis of approximations for maximizing submodular. Nips 2012 workshop on discrete optimization in machine learning. Constrained robust submodular optimization thomas powers 1, jeff bilmes, scott wisdom, david w. Submodular functions and optimization sciencedirect. Intuitively, a submodular function over the subsets demonstrates diminishing returns. Intelligent information gathering and submodular function optimization at ijcai 2009 by andreas krause and. Chapter ii submodular systems and base polyhedra pages 21125. Submodular functions occur in a variety of areas including graph theory, information theory, combinatorial optimization, stochastic processes and game theory. Optimization of submodular functions tutorial lecture ii jan vondrak1 1ibm almaden research center san jose, ca jan vondrak ibm almaden submodular optimization tutorial 1 24. Arpit agarwal 1 maximizing a submodular function in the last lecture we looked at maximization of a monotone submodular function, i. In particular, we present a variational approach to general logsubmodular and logsupermodular distributions based on sub and supergradients.
Entropy functions information sources entropy of the joint distribution conditional mutual information. Among these setfunctions, submodular functions play an important role, similar to convex functions on vector spaces. Think about functions applied to a set of sets of elements, you want to find t. Ieee transactions on network science and engineering 2. A problem instance consists of a distribution on 0,1n and a realvalued function on 0,1n that is nonnegative, monotone and submodular. Submodular programs unconstrained optimization 216 a minimizing submodular functions 217 b minimizing modular functions 223 7. Submodular functions capture a variety of discrete problems in machine learning, signal processing and computer vision. Combinatorial problems with submodular coupling in machine learning and computer vision.
In mathematics, a submodular set function also known as a submodular function is a set function whose value, informally, has the property that the difference in the incremental value of the function that a single element makes when added to an input set decreases as the size of the input set increases. In this paper, we rst describe tight modular bounds on. Our work combines central issues in optimization submodular functions and matroids. In the standard notions of learnability for submodular functions such as pac val84 and its generalization pmac bh11, the input is a collection of sampled sets and their function values, and the goal is to produce a surrogate that mimics the behavior of the function on samples drawn from. We provide algorithms for learning submodular functions, as well as lower bounds on their learnability. At rst glance it appears very similar to concavity, in other ways it resembles convexity. I in order to produce product e2eit is necessary to set up the machines needed to manufacture e, and this costs money. Submodular minimization nonsmooth convex optimization parallel algorithm for special case 3. Peng and yingyi bu in this lecture, the focus is on submodular function in combinatorial optimizations. Submodular optimization and machine learning part 1. Submodular functions have a natural diminishing returns property which makes them suitable.
A good knowledge of convex analysis is assumed see, e. One class, which contains the location problem, arises from matroids, another from the assignment problem and a third from boolean polynomials. Chapter v nonlinear optimization with submodular constraints pages 253283 download pdf. In chapter 11, we present other combinatorial optimization problems which can be partially solved using submodular analysis, such as submodularfunction maximization and the optimization of di. Mar 22, 2010 it also presents several examples of applying submodular function optimization to important machine learning problems, such as clustering, inference in probabilistic models and experimental design.
We demonstrate that many algorithms for submodular maximization may be viewed as special cases of a. There are specialized techniques for optimizing submodular functions. Overview of submodular minimization constrained submodular minimization constraint approximation hardness hardness ref vertex cover 2 ugc khot,regev 03 kunif. In this tutorial, the theory of submodular functions is presented, in a selfcontained way, with all results shown from. For example, a software company which produces a soft ware suite typically. An analysis of approximations for maximizing submodular set. Submodular functions are discrete functions that model laws of diminishing returns and enjoy numerous algorithmic applications. Constrained optimization 27, 7, 10, 16 minimize a submodular function f over a. Tutorial slides tutorials on submodularity in machine learning and computer vision at dagm 2012 and ecai 2012 by stefanie jegelka and andreas krause preliminary pdf invited tutorial intellgent optimization with submodular functions at lion 2012 by andreas krause. Convexity is a very natural property of various functions and domains occuring in such models. They have been used in many areas, including combinatorial optimization, machine learning, and economics.
A n algorithm for submodular functions on graphs 119 our purpose is to reverse some arrows of g so that the new digraph will be kstrongly connected. This includes algorithms for optimizing nonsubmodular functions via. This generalization enables to develop polynomial algorithms with guarantees by using ideas from continuous submodular optimization. Submodular continuous functions can already model many scenarios. At rst glance it appears very similar to concavity, in other ways it. Minimizing nonsubmodular functions with graph cuts a. Submodular functions often arise in various fields of operations research including discrete optimization, game theory, queueing theory and information theory.
Cargese workshop on combinatorial optimization, topic. Submodular functions play a vital role in modelling cost functions that naturally arise in many areas of discrete optimization. Submodular functions and optimization volume 58 annals. We carry out the first systematic investigation of inference in probabilistic models defined through submodular functions, generalizing regular binary markov random fields and determinantal point processes. Then g is an orientation of h with nonnegative costs on its arrows. Structure and scalability organized by stefanie jegelka, andreas krause, pradeep ravikumar, jeff bilmes. For this reason, an optimization problem which concerns optimizing a convex or concave function can also be described as the problem of maximizing or minimizing a submodular function subject to some constraints. In these areas, practical algorithms are a major concern. Citeseerx document details isaac councill, lee giles, pradeep teregowda. In this work we study submodular functions from a learning theoretic angle. Finally, anyone looking for an additional level of abstraction from submodular functions should look at murotas monograph on.
Submodular functions and convexity a submodular function is a function f. Creating submodular functions from other submodular functions is also possible, using sfo fn lincomb for nonnegative linear combinations, and sfo fn trunc for truncation. Convex analysis and optimization with submodular functions. Jan vondrak ibm almaden submodular functions and applications 7 28. Submodular function minimization grotschel, lovasz, schrijver 1981, 1988 iwata, fleischer, fujishige 2000 schrijver 2000 iwata 2003. Its a field dedicated to the optimization of submodular functions. Submodular optimization and approximation algorithms. I the setup cost isnonlinear, and it depends on which other products you choose to produce. We are interested in the information retrieval problem of.
Such functions are also referred to as additive or modular. Curvature and optimal algorithms for learning and minimizing. Submodular functions have properties which are very similar to convex and concave functions. Informally, the adaptive complexity of a problem is the minimal number of sequential rounds required to achieve. Custom submodular functions can be used either by inheriting from sfo fn, or by using the sfo fn wrapper function, which wraps a. Submodular function optimization file exchange matlab. Luckily, the submodular functions occurring in practice often have additional structure that can be exploited for practically efficient optimization algorithms. Submodularity has been an increasingly useful tool in machine learning in recent years. Submodular functions organized by samuel fiorini, gianpaolo oriolo, gautier stauffer and paolo ventura. This foundations and trendsr issue was typeset in latex using a class. This structure allows our algorithm to efficiently find the.
The rst class of submodular functions which was studied thoroughly was the class of matroid rank functions. In section 3 we present three classes of these problems. Submodular functions in combinatorial optimization date. Fast semidi erentialbased submodular function optimization in this paper, we address the issue of scalability and simultaneously draw connections across the apparent gap between minimization and maximization problems. Structured sparsityinducing norms relaxation of the penalization of supports by submodular functions. Algorithms for approximate minimization of the di erence. We provide algorithms for learning submodular functions, as well as lower bounds on. Submodular optimization is fast becoming a primary tool in. It also presents several examples of applying submodular function optimization to important machine learning problems, such as clustering, inference in probabilistic models and experimental design. Jul 27, 2015 submodular functions occur in a variety of areas including graph theory, information theory, combinatorial optimization, stochastic processes and game theory. Finally, anyone looking for an additional level of abstraction from submodular functions should look at murotas monograph on discrete convex analysis mur03. Edu university of washington, seattle, wa 98195, usa abstract we investigate a new optimization problem involving minimizing the ratio of two submodular rs functions. Optimization of submodular functions tutorial lecture ii.
Fast semidifferentialbased submodular function optimization. The unconstrained problem can be shown to be inapproximable if it is allowed to be negative. Submodular functions and optimization, volume 58 2nd edition. Submodular functions and optimization volume 58 annals of. Submodular function optimization file exchange matlab central. Unlike the case of minimization, maximizing a submodular functions is nphard even in the unconstrained setting. The adaptive complexity of maximizing a submodular function eric balkanski yaron singery abstract in this paper we study the adaptive complexity of submodular optimization. To motivate the representation of combinatorial optimization. Since the publication of the 1st edition of this book fifteen years ago, submodular functions have been showing further increasing importance in optimization, combinatorics, discrete mathematics, algorithmic computer science, and algorithmic economics, and. Machine learning and convex optimization with submodular.
Optimizing decomposable submodular functions institute for. Supermodularity and submodularity are also defined for functions defined over subsets of a larger set. Convex functions occur in many mathematical models in economy, engineering, and other sciencies. Since submodular functions have, in general, exponentialsize representation, 2 studied the problem of sketching submodular functions.
Yet, there are several interesting cases which are in general not drsubmodular, but can still be captured by a generalized notion. These functions have been studied under various models of computation. In this survey paper, we give overview on the fundamental properties of submodular functions and recent algorithmic devolopments of their minimization. This paper shows, by contrast, that for submodular optimization, mm algorithms have strong theoretical properties and empirically work very well. Maximizing a monotone submodular function subject to a. In this paper, we study the structure and learnability of. In machine learning, they emerge in different forms as the potential functions of graphical models, as the utility functions in active learning and sensing, in models of diversity, in. To draw a parallel to the booleanvalued case, a class of comparable breadth is the class of monotone boolean functions, which have been intensively studied 11, 9, 3. Harvey abstract this paper considers the problem of learning submodular functions.
Algorithms for optimizing the ratio of submodular functions graph gthat maps from any subset of the objects x uto the set of words x wcontained in the objects. Indeed, limited prior work has shown improved results for constrained minimization. They have been used in many areas, including combinatorial optimization, machine learning, and. The quality of any subset x umay be measured as the. It has widely been recognized that submodular functions play essential roles in efficiently solvable combinatorial optimization problems. In this paper, the theory of submodular functions is presented, in a selfcontained way, with all results proved from. Motivating \business school example i suppose that you manage a factory that is capable of making any one of a large nite set eof products. Learnability, structure, and optimization mariaflorina balcanyand nicholas j. They show that this process converges to a local minima, however the convergence rate is left as an open question. Algorithms for optimizing the ratio of submodular functions.
526 350 419 1303 1411 1444 144 455 1640 1045 860 594 43 1459 733 1222 93 763 1019 220 424 1552 957 1015 1130 1530 697 1598 1035 1628 1440 439 651 1243 1090 1080 425 452 1207 497 880 804 1372 95 380