Mfcttrf

C++20 constexpr std::copy optimizations for run-time

Is "The life is beautiful" incorrect or just very non-idiomatic?

I got kicked out from graduate school in the past. How do I include this on my CV?

Why is my Earth simulation slower than the reality?

Why can't an Airbus A330 dump fuel in an emergency?

Justifying the use of directed energy weapons

How to respectfully refuse to assist co-workers with IT issues?

Are modern clipless shoes and pedals that much better than toe clips and straps?

Why is Boris Johnson visiting only Paris & Berlin if every member of the EU needs to agree on a withdrawal deal?

Attaching a piece of wood to a necklace without drilling

Did it used to be possible to target a zone?

Why were movies shot on film shot at 24 frames per second?

How do I request a longer than normal leave of absence period for my wedding?

Rule based coloured background for labeling in QGIS

Why do all fields in a QFT transform like *irreducible* representations of some group?

How should I face my manager if I make a mistake because a senior coworker explained something incorrectly to me?

If the first law of thermodynamics ensures conservation of energy, why does it allow systems to lose energy?

What are some interesting features that are common cross-linguistically but don't exist in English?

In the MCU, why does Mjölnir retain its enchantments after Ragnarok?

Irish Snap: Variant Rules

What does どうかと思う mean?

Avoiding racist tropes in fantasy

See details of old sessions

Shouldn't the "credit score" prevent Americans from going deeper and deeper into personal debt?

What's the terminology for this alternative minimization algorithm?

The Remez AlgorithmAn eigenvalue algorithm to solve constrained quadratic form minimizationCan the Levenberg-Marquardt algorithm be used for minimization and not fittingHow to test convergence of an algorithm for constrained optimizationMinimization of The Blind Deconvolution FunctionalUnderstanding the conditions for which ADMM can be appliedPreconditioner for the GMRES method in the Uzawa algorithmObtaining a feasible solution for underdetermined system of linear equations satisfying inequality constraints

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

3

$begingroup$

Say the model is $F(x_1)G(x_2)Z(x_3) = y in mathbbR^N$, with $F,G,Z$ explicitly known, we are given observation of $y$ as $y_b in mathbbR^N$ to find the value of $x_1$, $x_2$, $x_3$ for each sample. Clearly this is an underdetermined inverse problem.

I have seen some iterative algorithm like this,

1. start with random number of $x_1$, $x_2, x_3$.


2. choose $x_1$, keep $x_2$, $x_3$ constant as change $x_1$ to $x_1^*$ such that

$$F(x_1^*)G(x_2)Z(x_3) = y_b$$

3. update $x_1 = x_1^*$




4. same thing for $x_2$, but fix $x_1$, $x_3$, and update $x_2 = x_2^*$.




5. repeat for $x_3$, and do the whole process several iterations until it doesn't change




6. final result is taken as the solution to the inverse problem.

I have seen this philosophy many times across different areas of computer science, just hope to know what is the big picture behind it. like ADMM?

optimization iterative-method admm

share|cite|improve this question

asked 8 hours ago

ArtificiallyIntelligenceArtificiallyIntelligence

15577 bronze badges

$endgroup$

add a comment | 

3

$begingroup$

Say the model is $F(x_1)G(x_2)Z(x_3) = y in mathbbR^N$, with $F,G,Z$ explicitly known, we are given observation of $y$ as $y_b in mathbbR^N$ to find the value of $x_1$, $x_2$, $x_3$ for each sample. Clearly this is an underdetermined inverse problem.

I have seen some iterative algorithm like this,

1. start with random number of $x_1$, $x_2, x_3$.


2. choose $x_1$, keep $x_2$, $x_3$ constant as change $x_1$ to $x_1^*$ such that

$$F(x_1^*)G(x_2)Z(x_3) = y_b$$

3. update $x_1 = x_1^*$




4. same thing for $x_2$, but fix $x_1$, $x_3$, and update $x_2 = x_2^*$.




5. repeat for $x_3$, and do the whole process several iterations until it doesn't change




6. final result is taken as the solution to the inverse problem.

I have seen this philosophy many times across different areas of computer science, just hope to know what is the big picture behind it. like ADMM?

optimization iterative-method admm

share|cite|improve this question

asked 8 hours ago

ArtificiallyIntelligenceArtificiallyIntelligence

15577 bronze badges

$endgroup$

add a comment | 

$begingroup$

Say the model is $F(x_1)G(x_2)Z(x_3) = y in mathbbR^N$, with $F,G,Z$ explicitly known, we are given observation of $y$ as $y_b in mathbbR^N$ to find the value of $x_1$, $x_2$, $x_3$ for each sample. Clearly this is an underdetermined inverse problem.

I have seen some iterative algorithm like this,

1. start with random number of $x_1$, $x_2, x_3$.


2. choose $x_1$, keep $x_2$, $x_3$ constant as change $x_1$ to $x_1^*$ such that

$$F(x_1^*)G(x_2)Z(x_3) = y_b$$

3. update $x_1 = x_1^*$




4. same thing for $x_2$, but fix $x_1$, $x_3$, and update $x_2 = x_2^*$.




5. repeat for $x_3$, and do the whole process several iterations until it doesn't change




6. final result is taken as the solution to the inverse problem.

I have seen this philosophy many times across different areas of computer science, just hope to know what is the big picture behind it. like ADMM?

optimization iterative-method admm

share|cite|improve this question

asked 8 hours ago

ArtificiallyIntelligenceArtificiallyIntelligence

15577 bronze badges

$endgroup$

share|cite|improve this question

asked 8 hours ago

ArtificiallyIntelligenceArtificiallyIntelligence

15577 bronze badges

share|cite|improve this question

asked 8 hours ago

ArtificiallyIntelligenceArtificiallyIntelligence

15577 bronze badges

asked 8 hours ago

ArtificiallyIntelligenceArtificiallyIntelligence

15577 bronze badges

asked 8 hours ago

ArtificiallyIntelligenceArtificiallyIntelligence

15577 bronze badges

1 Answer
1

active

oldest

votes

5

$begingroup$

A general name for this approach is "Block Coordinate Descent." It's important to understand that convergence isn't guaranteed without additional hypotheses.

ADMM is not simply block coordinate descent- it's a more complicated method that is optimizing with respect to primal variables $x$ and $z$ in each iteration and then adjusting the Lagrange multiplier and penalty parameter.

share|cite|improve this answer

answered 7 hours ago

Brian BorchersBrian Borchers

13.9k11 gold badge2222 silver badges5353 bronze badges

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  just found in a 1977 paper, it is called nonlinear Gauss Seidel method. oh yeah, now it looks more familiar.
  $endgroup$
  – ArtificiallyIntelligence
  1 hour ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "363"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fscicomp.stackexchange.com%2fquestions%2f33297%2fwhats-the-terminology-for-this-alternative-minimization-algorithm%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

5

$begingroup$

A general name for this approach is "Block Coordinate Descent." It's important to understand that convergence isn't guaranteed without additional hypotheses.

ADMM is not simply block coordinate descent- it's a more complicated method that is optimizing with respect to primal variables $x$ and $z$ in each iteration and then adjusting the Lagrange multiplier and penalty parameter.

share|cite|improve this answer

answered 7 hours ago

Brian BorchersBrian Borchers

13.9k11 gold badge2222 silver badges5353 bronze badges

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  just found in a 1977 paper, it is called nonlinear Gauss Seidel method. oh yeah, now it looks more familiar.
  $endgroup$
  – ArtificiallyIntelligence
  1 hour ago
  

  
  
  
  

add a comment | 

5

$begingroup$

A general name for this approach is "Block Coordinate Descent." It's important to understand that convergence isn't guaranteed without additional hypotheses.

ADMM is not simply block coordinate descent- it's a more complicated method that is optimizing with respect to primal variables $x$ and $z$ in each iteration and then adjusting the Lagrange multiplier and penalty parameter.

share|cite|improve this answer

answered 7 hours ago

Brian BorchersBrian Borchers

13.9k11 gold badge2222 silver badges5353 bronze badges

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  just found in a 1977 paper, it is called nonlinear Gauss Seidel method. oh yeah, now it looks more familiar.
  $endgroup$
  – ArtificiallyIntelligence
  1 hour ago
  

  
  
  
  

add a comment | 

$begingroup$

A general name for this approach is "Block Coordinate Descent." It's important to understand that convergence isn't guaranteed without additional hypotheses.

ADMM is not simply block coordinate descent- it's a more complicated method that is optimizing with respect to primal variables $x$ and $z$ in each iteration and then adjusting the Lagrange multiplier and penalty parameter.

share|cite|improve this answer

answered 7 hours ago

Brian BorchersBrian Borchers

13.9k11 gold badge2222 silver badges5353 bronze badges

$endgroup$

share|cite|improve this answer

answered 7 hours ago

Brian BorchersBrian Borchers

13.9k11 gold badge2222 silver badges5353 bronze badges

answered 7 hours ago

Brian BorchersBrian Borchers

13.9k11 gold badge2222 silver badges5353 bronze badges

answered 7 hours ago

Brian BorchersBrian Borchers

13.9k11 gold badge2222 silver badges5353 bronze badges