Mfcttrf

Is Odin inconsistent about the powers of Mjolnir?

Best way to explain to my boss that I cannot attend a team summit because it is on Rosh Hashana or any other Jewish Holiday

What can make Linux so unresponsive?

What is the German idiom or expression for when someone is being hypocritical against their own teachings?

Can I enter a rental property without giving notice if I'm afraid a tenant may be hurt?

How do these cubesats' whip antennas work?

Why should I "believe in" weak solutions to PDEs?

How to help new students accept function notation

12V lead acid charger with LM317 not charging

Is DC heating faster than AC heating?

How to draw a flow chart?

Did silent film actors actually say their lines or did they simply improvise “dialogue” while being filmed?

Can a Hogwarts student refuse the Sorting Hat's decision?

Onenote - Reducing Storage Footprint on PC

Will a paper be retracted if a flaw in released software code invalidates its central idea?

Why don't the open notes matter in guitar chords?

How to realistically deal with a shield user?

Print only the last three columns from file

Does this smartphone photo show Mars just below the Sun?

What does VB stand for?

Using command line how to open a specific section of GUI System Preferences?

Decode a variable-length quantity

Did Apollo leave poop on the moon?

Count number of occurences of particular numbers in list

Why ReLU function is not differentiable at 0?

Why is this function differentiable?Determining values where a function is not differentiableIs it possible that $f$ is differentiable?Why can a discontinuous function not be differentiable?Why is this function not differentiable at $(0,0)$Why is $f(x)$ not differentiable at $0$?How to prove the given function is not differentiable analytically?Why isn't this function differentiable?Finding where a function is not differentiableIs this non monotonic function differentiable at $x=1$?

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

3

$begingroup$

I'm kind of rusty in calculus.

Why the ReLU function is not differentiable at $f(0)$?

$
f(x) =
left{beginmatrix
0 quad if ; x leq 0\
x quad if ; x > 0
endmatrixright.
$

calculus

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

$endgroup$

add a comment | 

3

$begingroup$

I'm kind of rusty in calculus.

Why the ReLU function is not differentiable at $f(0)$?

$
f(x) =
left{beginmatrix
0 quad if ; x leq 0\
x quad if ; x > 0
endmatrixright.
$

calculus

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

$endgroup$

add a comment | 

$begingroup$

I'm kind of rusty in calculus.

Why the ReLU function is not differentiable at $f(0)$?

$
f(x) =
left{beginmatrix
0 quad if ; x leq 0\
x quad if ; x > 0
endmatrixright.
$

calculus

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

$endgroup$

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

2 Answers
2

active

oldest

votes

5

$begingroup$

If you look at $x > 0$, or the righthand derivative, $$fracdfdx = fracddx x = 1$$ for all $x$.

If you look at $x le 0$, or the lefthand derivative, $$fracdfdx = fracddx 0 = 0$$ for all $x$.

Since $x = 0$ is the "break" point, the lefthand and righthand derivatives are not the same, and thus, the derivative is not defined at $x = 0$.

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

$endgroup$

add a comment | 

4

$begingroup$

Because it has a cusp at $0$ (a sharp corner), so it doesn't have a well defined tangent line; think about it, a tangent line is one which touches the graph only at one point. You can imagine many lines going through $(0, 0)$, so there are many possible tangent lines.

More formally, we have to investigate the limit

$$lim_h to 0 dfracf(0+h) - f(0)h$$

This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get

$$lim_h to 0^+ dfrac h-0h=1.$$

While if you let $h$ approach $0$ from the left, that limit

$$lim_h to 0^- dfrac 0-0h=0.$$

Therefore the limit does not exist, so the function is not differentiable at $0$.

share|cite|improve this answer

edited 7 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

$endgroup$

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3316444%2fwhy-relu-function-is-not-differentiable-at-0%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

5

$begingroup$

If you look at $x > 0$, or the righthand derivative, $$fracdfdx = fracddx x = 1$$ for all $x$.

If you look at $x le 0$, or the lefthand derivative, $$fracdfdx = fracddx 0 = 0$$ for all $x$.

Since $x = 0$ is the "break" point, the lefthand and righthand derivatives are not the same, and thus, the derivative is not defined at $x = 0$.

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

$endgroup$

add a comment | 

5

$begingroup$

If you look at $x > 0$, or the righthand derivative, $$fracdfdx = fracddx x = 1$$ for all $x$.

If you look at $x le 0$, or the lefthand derivative, $$fracdfdx = fracddx 0 = 0$$ for all $x$.

Since $x = 0$ is the "break" point, the lefthand and righthand derivatives are not the same, and thus, the derivative is not defined at $x = 0$.

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

$endgroup$

add a comment | 

$begingroup$

If you look at $x > 0$, or the righthand derivative, $$fracdfdx = fracddx x = 1$$ for all $x$.

If you look at $x le 0$, or the lefthand derivative, $$fracdfdx = fracddx 0 = 0$$ for all $x$.

Since $x = 0$ is the "break" point, the lefthand and righthand derivatives are not the same, and thus, the derivative is not defined at $x = 0$.

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

$endgroup$

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

4

$begingroup$

Because it has a cusp at $0$ (a sharp corner), so it doesn't have a well defined tangent line; think about it, a tangent line is one which touches the graph only at one point. You can imagine many lines going through $(0, 0)$, so there are many possible tangent lines.

More formally, we have to investigate the limit

$$lim_h to 0 dfracf(0+h) - f(0)h$$

This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get

$$lim_h to 0^+ dfrac h-0h=1.$$

While if you let $h$ approach $0$ from the left, that limit

$$lim_h to 0^- dfrac 0-0h=0.$$

Therefore the limit does not exist, so the function is not differentiable at $0$.

share|cite|improve this answer

edited 7 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

$endgroup$

add a comment | 

4

$begingroup$

Because it has a cusp at $0$ (a sharp corner), so it doesn't have a well defined tangent line; think about it, a tangent line is one which touches the graph only at one point. You can imagine many lines going through $(0, 0)$, so there are many possible tangent lines.

More formally, we have to investigate the limit

$$lim_h to 0 dfracf(0+h) - f(0)h$$

This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get

$$lim_h to 0^+ dfrac h-0h=1.$$

While if you let $h$ approach $0$ from the left, that limit

$$lim_h to 0^- dfrac 0-0h=0.$$

Therefore the limit does not exist, so the function is not differentiable at $0$.

share|cite|improve this answer

edited 7 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

$endgroup$

add a comment | 

$begingroup$

Because it has a cusp at $0$ (a sharp corner), so it doesn't have a well defined tangent line; think about it, a tangent line is one which touches the graph only at one point. You can imagine many lines going through $(0, 0)$, so there are many possible tangent lines.

More formally, we have to investigate the limit

$$lim_h to 0 dfracf(0+h) - f(0)h$$

This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get

$$lim_h to 0^+ dfrac h-0h=1.$$

While if you let $h$ approach $0$ from the left, that limit

$$lim_h to 0^- dfrac 0-0h=0.$$

Therefore the limit does not exist, so the function is not differentiable at $0$.

share|cite|improve this answer

edited 7 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

$endgroup$

share|cite|improve this answer

edited 7 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

edited 7 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

edited 7 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges